arXiv AI
Jun 26
The Capability Frontier: Benchmarks Miss 82% of Model Performance
arXiv:2606. 26836v1 Announce Type: new Abstract: Existing benchmarks typically report accuracy for a single model on a single run.
Prescriptive Scaling Reveals the Evolution of Language Model Capabilities
arXiv:2602. 15327v2 Announce Type: replace-cross Abstract: Machine learning model performance improvements tend to arise from competition and application.
Related stories
arXiv AI
Aug 7
Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning
arXiv:2604. 01170v2 Announce Type: replace-cross Abstract: While test-time scaling has enabled large language models to solve highly difficult tasks, state-of-the-art results come at exorbitant compute costs.
arXiv AI
Aug 5
Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility
arXiv:2608. 04001v1 Announce Type: cross Abstract: Large language models can solve substantially harder reasoning problems with more inference-time compute.
arXiv AI
Jul 28
DeepLook: Deeper Thinking with Lookahead
arXiv:2607. 22602v1 Announce Type: new Abstract: Inference-time scaling has emerged as a powerful paradigm for improving large language model reasoning, often delivering larger gains on difficult reasoning tasks than parameter scaling alone.
arXiv AI
Jul 29
Bridging Compute- and Data-Optimal Pretraining
arXiv:2607. 25271v1 Announce Type: cross Abstract: Classical compute-optimal scaling laws assume an unbounded supply of fresh pretraining data, yet pretraining is increasingly entering a regime in which compute grows faster than the availability of high-quality data.
arXiv AI
Jun 17
How Inference Compute Shapes Frontier LLM Evaluation
arXiv:2606. 17930v1 Announce Type: new Abstract: AI evaluations are shifting toward harder tasks that benefit from longer trajectories involving tool use and iterative problem solving.
arXiv AI
Jun 3
q0: Primitives for Hyper-Epoch Pretraining
arXiv:2606. 03938v1 Announce Type: cross Abstract: Multi-epoch training is becoming the standard now that compute is growing faster than the supply of high-quality text.
arXiv AI
Aug 11
Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute
arXiv:2608. 09351v1 Announce Type: cross Abstract: Test-time scaling improves LLM accuracy but multiplies inference cost, making the accuracy gained per unit of compute the metric that matters in deployment.
arXiv AI
Jun 10
Using Probabilistic Programs to Train Inductive Reasoning in Large Language Models
arXiv:2606. 09856v1 Announce Type: cross Abstract: Post-training Large Language Models (LLMs) for reasoning typically focuses on deductive tasks such as mathematics and coding where correctness is verifiable.
Hugging Face Trending Papers
Jul 28
Bridging Compute- and Data-Optimal Pretraining
Classical compute-optimal scaling laws assume an unbounded supply of fresh pretraining data, yet pretraining is increasingly entering a regime in which compute grows faster than the availability of high-quality data. We propose Compute-Data (CD) scaling laws, a unified framework that bridges compute-optimal scaling, where data scales freely with compute, and data-optimal scaling, where the corpus is fixed while compute can grow without bound.
arXiv AI
Jul 8
PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages
arXiv:2607. 05992v1 Announce Type: cross Abstract: Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites.
arXiv AI
Jun 10
Unifying Data, Memory, and Compute Efficiency in LLM training: A Survey
arXiv:2606. 10706v1 Announce Type: cross Abstract: Resource constraints increasingly determine what can be trained, fine-tuned, and deployed in large language models (LLMs), yet efficiency is often studied through isolated techniques rather than as an interacting system of limits.