Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

17,067 stories · RSS feed

arXiv AI
Jul 7

Spectral Rewiring for Exploration, Purification, and Model Merging

arXiv:2607. 03065v1 Announce Type: cross Abstract: Reinforcement learning has become a standard post-training recipe for large language models, but dense full-parameter updates create two deployment-relevant bottlenecks: suppressed reasoning performance, often reflected by premature saturation of test-time scaling, and interference when consolidating multiple capabilities through multi-domain training or model merging.

By Zhilong Zhang, Hongli Yu, Huan-ang Gao, Hanlin Wu, Yuxuan Song, Wei-Ying Ma, Ya-Qin Zhang, Hao Zhou
arXiv Machine Learning
Jul 7

Graph Neural Networks for the Graphical Bootstrap

arXiv:2607. 03109v1 Announce Type: cross Abstract: We study a graph classification problem involving over 20 million graphs, arising from high-order perturbative computations of correlators in planar $\mathcal{N}=4$ super-Yang--Mills, a model closely related to the theory of the strong nuclear force.

By Rigers Aliaj, Gabriele Dian, Reza Doobary, Paul Heslop
arXiv Machine Learning
Jul 7

Memisis: Orchestrating and Evaluating Synthetic Data for Tabular Health Datasets

arXiv:2605. 17758v2 Announce Type: replace Abstract: Synthetic data is widely used in healthcare to create datasets that preserve statistical properties of real data without exposing sensitive patient information.

By Nitish Nagesh, Pengbao Zhou, Atchuth Naveen Chilaparasetti, Yajat Nagaraj Kiran, Tu Nguyen, Arshia Harish Puthran, Muhjaazee Love, Aadi Sharma, Mahdi Bagheri, Ian Harris, Amir M. Rahmani
arXiv AI
Jul 7

Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models

arXiv:2607. 05199v1 Announce Type: new Abstract: Physics reasoning fails structurally in small language models: an error at any step propagates forward, corrupting every inference that follows.

By Raj Jaiswal, Dhruv Jain, Rishabh Dhawan, Sree Krishna Uppalapati, Shin'ichi Satoh, Tanuja Ganu, Rajiv Ratn Shah
arXiv Machine Learning
Jul 7

Latent Programming Horizons in Coding Agents

arXiv:2607. 05188v1 Announce Type: new Abstract: A coding agent solving a software-engineering task spends dozens of steps reasoning, editing code, and running tests, yet little is known about what the underlying language model internally represents about the program it is working on.

By Andr\'e Silva, Han Tu, Martin Monperrus
arXiv Machine Learning
Jul 7

Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe

arXiv:2605. 03677v2 Announce Type: replace Abstract: On-policy distillation (OPD) has recently emerged as an effective post-training paradigm for consolidating the capabilities of specialized expert models into a single student model.

By Wenjin Hou, Shangpin Peng, Weinong Wang, Zheng Ruan, Yue Zhang, Zhenglin Zhou, Mingqi Gao, Yifei Chen, Kaiqi Wang, Hongming Yang, Chengquan Zhang, Zhuotao Tian, Han Hu, Yi Yang, Fei Wu, Hehe Fan