← Back to all news
arXiv AI By Jeremy Canale

Toward Alignment Scaling Laws: A Framework and First Preregistered Measurements

Read the original on arXiv AI →

The paper proposes a framework for studying how alignment difficulty scales with model size, treating alignment as a family of measurable scaling relations for different risk categories. It introduces operational definitions of burden, distinguishes observed, audited, and true alignment, and proves that the largest exponent among corrected risks determines long‑run behavior. The authors preregistered and applied the protocol to public data, finding scaling exponents for attack success, truthfulness, dispositions, sycophancy, and a backdoor, and released interactive browser games to illustrate the laws.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Jun 30

Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models

arXiv:2606. 30627v1 Announce Type: cross Abstract: Conservative offline training is widely advocated as a safe foundation for subsequent online adaptation: if a policy stays close to well-supported behaviour, the argument goes, it is less likely to exploit imperfections in a learned reward model.

By Subramanyam Sahoo, Aman Chadha, Vinija Jain, Divya Chaudhary
reinforcement-learningsafety
More like this →
arXiv AI
Oct 2

False Floors: LLM Safety Routing Evaluations Break Under Distribution Shift

The paper examines safety routers—systems that route user requests to different language models—and finds that their performance degrades significantly when evaluated under distribution shift. In standard benchmarks, routers appear effective because the best single model is chosen from the same evaluation data, but when the data distribution changes, the routing advantage diminishes or disappears. The study quantifies this bias across multiple safety corpora, showing that routers offer little benefit under realistic shift conditions and that recognition‑based defenses can be undermined by attackers who know the model being used.

By Amit Singh Bhatti, Vishal Vaddina
llmsbenchmarkssafety
More like this →
arXiv Machine Learning
Sep 23

Practical Scaling Laws: Converting Compute into Performance in a Data-Constrained World

The paper introduces a new closed‑form scaling law that extends Chinchilla’s original formula to handle data‑constrained regimes. It decomposes loss into undercapacity, undertraining, and overfitting components, saturating between an irreducible loss and an uninformed baseline. The authors validate the model on diverse architectures and domains, achieving state‑of‑the‑art RMSE across multiple LLM scaling‑law grids and enabling cost‑aware training allocations.

By Christopher M. Bryant, Hao Liu
llmsbenchmarks
More like this →