The paper investigates whether a looped estimator—one that repeatedly applies a single fitted operator with shared parameters—can enhance statistical accuracy while staying within a fixed parameter budget. It establishes upper and lower bounds on squared Hellinger risk for looped sieve maximum likelihood and compares them to the untied counterpart, revealing a tradeoff between parameter sharing, iteration count, and accuracy. For models with known H"older smoothness, looped residual feedforward networks and a post‑layer‑normalized Transformer achieve minimax polynomial rates with a fixed number of bounded real parameters, and under certain conditions the looped estimator’s worst‑case risk vanishes as sample size grows, outperforming the untied approach.
arXiv:2606. 06772v2 Announce Type: replace-cross Abstract: Characterizing the optimization dynamics and statistical performance of over-parameterized deep neural networks (DNNs) remains a central challenge in understanding the remarkable success of deep learning.
arXiv:2607. 23454v1 Announce Type: new Abstract: Data-driven remaining useful life (RUL) prediction requires complete degradation trajectories for training, yet such run-to-failure data are scarce and expensive.
arXiv:2609.25576v1 Announce Type: cross Abstract: We study the estimation of a $K$-dimensional simplex from $N$ i.i.d.\ points sampled uniformly from its interior; the observations are convex combina...
arXiv:2609.40316v1 Announce Type: cross Abstract: Looped transformers and Mixture-of-Experts (MoE) offer complementary routes to efficient scaling: recurrence increases computational depth at fixed p...
arXiv:2606. 00605v1 Announce Type: new Abstract: Transformers have achieved remarkable success across a wide range of applications, and a growing body of work suggests that part of their strength comes from their ability to learn and execute algorithmic procedures.
Looped transformers and Mixture-of-Experts (MoE) offer complementary routes to efficient scaling: recurrence increases computational depth at fixed parameters, while MoE sparsity expands total capacit...
arXiv:2512. 22088v3 Announce Type: replace-cross Abstract: The scaling law, a cornerstone of Large Language Model (LLM) development, predicts improvements in model performance with increasing computational resources.
arXiv:2509. 09078v3 Announce Type: replace-cross Abstract: Given-data methods for variance-based sensitivity analysis have significantly advanced the feasibility of Sobol' index computation for computationally expensive models and models with many inputs.
arXiv:2606. 17649v1 Announce Type: cross Abstract: The high cost of fine-tuning LLMs poses a significant economic barrier; pre-hoc performance prediction offers a critical solution to substantially reduce this expense.
arXiv:2603. 20388v2 Announce Type: replace-cross Abstract: We derive the asymptotic risk function of regularized empirical risk minimization (ERM) estimators tuned by $n$-fold cross-validation (CV).
arXiv:2605. 27991v2 Announce Type: replace-cross Abstract: Gradient-flow optimization is usually viewed as an algorithmic procedure for minimizing empirical loss, with training duration selected by validation or heuristic early-stopping rules.
The paper introduces LOOM, a method for scaling looped mixture‑of‑experts (MoE) Transformers beyond the typical two‑loop limit. LOOM addresses two key obstacles: it stabilizes deep recurrence by bounding residual variance and re‑injecting the input embedding, and it prevents expert selection collapse by using per‑loop routers and a looping residual to maintain computational diversity. Experiments on 100 M–1.7 B parameter models show stable scaling to 9–12 loops, with significant perplexity reductions and zero‑shot accuracy gains under near‑iso‑FLOP conditions.