arXiv Machine Learning

When Do Local Score Models Extrapolate Across Size? A Diagnostic Theory and Benchmark

arXiv:2606. 09705v1 Announce Type: new Abstract: Scientific generative modeling often requires size transfer, where models trained on small systems are evaluated on larger ones.

arXiv Machine Learning
Sep 24

Localized Diffusion Models

The paper introduces localized diffusion models, which exploit locality structure—sparse conditional dependencies among target variables—to reduce the dimensionality of the score function. By training a localized neural network with a localized score matching loss, the authors demonstrate that diffusion models can achieve dimension‑independent error bounds, balancing statistical and localization errors with a moderate radius. This approach also enables parallel training, potentially improving efficiency for large‑scale applications.

By Georg A. Gottwald, Shuigen Liu, Youssef Marzouk, Sebastian Reich, Xin T. Tong
arXiv Machine Learning
Jun 16

Amortized mean-shift interacting particles

arXiv:2606. 15871v1 Announce Type: cross Abstract: Bayesian inference for inverse problems is run to evaluate integrals -- posterior expectations, tail probabilities, and risks -- across a stream of observations.

By Ali Siahkoohi
Hugging Face Trending Papers
Aug 12

Small-Scale Experiments: Are We There Yet?

Scaling laws promised cost-effective experiments; six years later, they have yet to fully deliver. Instead, researchers have found them unreliable at small scales (starting at 4M parameters) and concluded that sizable models cannot be avoided.

arXiv AI
Jul 16

DeepLoop: Depth Scaling for Looped Transformers

arXiv:2607. 13491v1 Announce Type: cross Abstract: Looped Transformers scale sequential computation by applying a compact stack of physical blocks for multiple rounds, increasing unrolled depth without increasing stored parameters.

By Shuzhen Li, Yifan Zhang, Jiacheng Guo, Quanquan Gu, Mengdi Wang