arXiv AI

Item Response Scaling Laws: A Measurement Theory Approach for Efficient and Generalizable Neural Scaling Estimation

arXiv:2606. 07616v1 Announce Type: cross Abstract: Scaling laws provide a fundamental framework for understanding the performance of Language Models (LMs), yet deriving them requires prohibitively expensive evaluations across thousands of checkpoints or millions of inference samples.

arXiv AI

Jul 20

Latency-Response Theory Model: Evaluating Large Language Models via Response Accuracy and Chain-of-Thought Length

arXiv:2512. 07019v3 Announce Type: replace-cross Abstract: The proliferation of Large Language Models (LLMs) necessitates valid evaluation methods to provide guidance for both downstream applications and actionable future improvements.

By Zhiyu Xu, Jia Liu, Yixin Wang, Yuqi Gu

llms benchmarks

arXiv AI

Jun 9

Prescriptive Scaling Reveals the Evolution of Language Model Capabilities

arXiv:2602. 15327v2 Announce Type: replace-cross Abstract: Machine learning model performance improvements tend to arise from competition and application.

By Hanlin Zhang, Jikai Jin, Vasilis Syrgkanis, Sham Kakade

llms benchmarks

arXiv AI

Jul 1

Optimal Self-Consistency for Efficient Reasoning with Large Language Models

arXiv:2511. 12309v2 Announce Type: replace-cross Abstract: Self-consistency (SC) is a widely used test-time inference technique for improving performance in chain-of-thought reasoning.

By Austin Feng, Marius Alonso, Ambroise Odonnat, Vasilii Feofanov, Ievgen Redko

llms benchmarks

arXiv AI

Jun 2

ProbeScale: Probing Analysis to Optimize Neural Scaling Laws for Efficient Small Language Model Inference

arXiv:2606. 01806v1 Announce Type: cross Abstract: Small Language Models (SLMs) offer a balance between capability and computational feasibility.

By Sourav Das

llms

arXiv Machine Learning

Jun 4

A Latent Variable Framework for Scaling Laws in Large Language Models

arXiv:2512. 06553v2 Announce Type: replace-cross Abstract: We propose a statistical framework built on latent variable modeling for scaling laws of large language models (LLMs).

By Peiyao Cai, Chengyu Cui, Felipe Maia Polo, Seamus Somerstep, Leshem Choshen, Mikhail Yurochkin, Yuekai Sun, Kean Ming Tan, Gongjun Xu

llms benchmarks

arXiv Machine Learning

Jul 23

Scaling Laws for Hypernetwork-Based Knowledge Injection in Large Language Models

arXiv:2607. 19604v1 Announce Type: cross Abstract: Injecting factual knowledge into large language models (LLMs) reliably and at scale remains an open challenge.

By Nischay Dhankhar, Dos Baha, Abulhair Saparov

llms fine-tuning

arXiv AI

Aug 11

Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute

arXiv:2608. 09351v1 Announce Type: cross Abstract: Test-time scaling improves LLM accuracy but multiplies inference cost, making the accuracy gained per unit of compute the metric that matters in deployment.

By Nikita Kozodoi, Zainab Afolabi, Jack Butler

llms nlp multimodal

arXiv AI

Aug 5

Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility

arXiv:2608. 04001v1 Announce Type: cross Abstract: Large language models can solve substantially harder reasoning problems with more inference-time compute.

By Mohsen Hariri, Weicong Chen, Nahal Shahini, Vikash Singh, Kai Ye, Amirhossein Samandar, Debargha Ganguly, Sreehari Sankar, Yanyan Zhang, Shouren Wang, Jerry Peng, Biyao Zhang, Michael Hinczewski, Vipin Chaudhary

llms benchmarks

arXiv AI

Jun 2

When Data Is Scarce: Scaling Sparse Language Models with Repeated Training

arXiv:2606. 01155v1 Announce Type: cross Abstract: Scaling laws for dense LLMs under infinite data are well explored, but how sparsity interacts with limited data is not.

By Boqian Wu, Qiao Xiao, Patrik Okanovic, Tomasz Sternal, Maurice van Keulen, Mykola Pechenizkiy, Elena Mocanu, Torsten Hoefler, Decebal Constantin Mocanu

llms efficiency

arXiv AI

Jul 29

Laplace-PSN-IRT: Uncertainty Quantification for Neural Item Response Theory Models of LLM Benchmarks

arXiv:2607. 25257v1 Announce Type: cross Abstract: Item Response Theory (IRT) has recently been proposed as a framework for evaluating large language model (LLM) benchmarks by separating a model's latent ability from the properties of individual benchmark items.

By Juan Francisco, Mandujano Reyes

llms benchmarks

arXiv Machine Learning

Aug 12

UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking

arXiv:2603. 24226v4 Announce Type: replace-cross Abstract: Recent advances in Large Language Models (LLMs) have inspired a surge of scaling research in industrial search, advertising, and recommendation systems.

By Liren Yu, Caiyuan Li, Feiyi Dong, Tao Zhang, Zhixuan Zhang, Dan Ou, Haihong Tang, Bo Zheng

llms

arXiv AI

Jul 17

Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect

arXiv:2607. 14111v1 Announce Type: cross Abstract: Can small language models detect and report on perturbations their own internal activations?

By Ely Hahami, Ishaan Sinha, Lavik Jain

llms fine-tuning benchmarks safety

Related stories