The paper introduces a new closed‑form scaling law that extends Chinchilla’s original formula to handle data‑constrained regimes. It decomposes loss into undercapacity, undertraining, and overfitting components, saturating between an irreducible loss and an uninformed baseline. The authors validate the model on diverse architectures and domains, achieving state‑of‑the‑art RMSE across multiple LLM scaling‑law grids and enabling cost‑aware training allocations.
By Christopher M. Bryant, Hao Liu
arXiv:2608.23573v1 Announce Type: new
Abstract: A trained transformer's weight magnitudes can be summarized by a two-parameter Weibull distribution whose shape $k \approx 1.2$ is stable across layers...
By Tiexin Ding
The paper proposes a framework for studying how alignment difficulty scales with model size, treating alignment as a family of measurable scaling relations for different risk categories. It introduces operational definitions of burden, distinguishes observed, audited, and true alignment, and proves that the largest exponent among corrected risks determines long‑run behavior. The authors preregistered and applied the protocol to public data, finding scaling exponents for attack success, truthfulness, dispositions, sycophancy, and a backdoor, and released interactive browser games to illustrate the laws.
By Jeremy Canale
arXiv:2609.27581v1 Announce Type: new
Abstract: Step Law gives power-law formulas for the optimal peak learning rate eta* and batch size B* when pre-training language models. It was calibrated on mod...
By Egor Romanyukov, Timofey Novikov, Timur Shokarov, Elizaveta Zorkina, Anastasia Palienko, Stepan Dergachev
arXiv:2605.08541v3 Announce Type: replace
Abstract: Neural scaling laws approximate a language model's loss as a power-law function of parameter count $N$ and token count $D$. Following Chinchilla-st...
By Joshua Shay Kricheli, Alexander Lawrence Reid, Soumajyoti Sarkar, Venkata Gandikota, Paulo Shakarian
The paper evaluates a narrow implementation of domain‑specific curvature called HyperDG, which rescales Euclidean encoder features during training with learned positive scalars and uses a standard linear classifier at inference. Across six conditions on four benchmark datasets (PACS, VLCS, Office Home, TerraIncognita) and 384 runs, the method’s macro performance is 1.61 percentage points below native DomainBed ERM, with no statistically significant difference after adjustment. The study concludes that this scalar‑scaling approach does not provide evidence of improvement over ERM and offers a bounded negative result for this specific training package.