arXiv:2602. 15327v2 Announce Type: replace-cross Abstract: Machine learning model performance improvements tend to arise from competition and application.
By Hanlin Zhang, Jikai Jin, Vasilis Syrgkanis, Sham Kakade
Predictable Scaling (PS) is a framework that uses small‑scale training proxies to forecast full‑scale model performance, reducing the need for expensive GPU‑hourly runs. By applying composable scaling functions and curve fitting with uncertainty estimates, PS can rank candidate changes accurately across different hypothesis types. Experiments on two ranking models show PS achieves 79–88% compute savings while maintaining correct ranking of actionable hypotheses.
By Aishwarya Wesanekar, Sheng-Tao Yang, Jiawei Zhang, Flavio Sales Truzzi, Chen Chen, Nader Beigiparast, Shengyuan Huang, Roshanak Zilouchian Moghaddam, Vitor Cid, Colin Clement, Vinayak Thapliyal, Sebastian Ament, Jihao Andreas Lin, Vincent Huang, Kai Han, Boyi Jiang, Wenlin Chen, Chaorong Chen, Ashwin Kumar, Maximilian Balandat, David Eriksson, Yifei Zhang, Pan Zhong
arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.
By Selen Erkan, Bastian Boll, Kristian Kersting, Bj\"orn Deiseroth, Letitia Parcalabescu
arXiv:2606. 07616v1 Announce Type: cross Abstract: Scaling laws provide a fundamental framework for understanding the performance of Language Models (LMs), yet deriving them requires prohibitively expensive evaluations across thousands of checkpoints or millions of inference samples.
By Sang Truong, Yuheng Tu, Rylan Schaeffer, Sanmi Koyejo
Osprey is a target‑agnostic pre‑training method that bootstraps draft models for speculative decoding from existing small language models. By pruning to a shallow backbone, restoring language‑modeling capability with next‑token pretraining, and adapting via vocabulary alignment and distillation, Osprey reduces per‑target work to a lightweight adaptation step. Experiments show that a single Osprey backbone improves mean acceptance length by up to 22.7% and increases tokens per second by 17.5% across several large target models, especially on out‑of‑domain and multilingual data.
By Fengxiang Bie, Yuqing Jian, Yifan Yu, Zhongzhu Zhou, Zelei Shao, Ben Athiwaratkun, Shuaiwen Leon Song, Chenfeng Xu, Xiaoxia Wu, Tianyi Zhang
arXiv:2607. 23377v1 Announce Type: cross Abstract: The largest machine learning models in particle physics are also the most expensive to train, yet the return on scaling a given architecture cannot be estimated before that compute is spent.
By Jan-Lucas Uslu, Benjamin Nachman, Christopher Re