arXiv Machine Learning

Efficient Sequential Evaluation of Large Language Models

arXiv:2607. 17409v1 Announce Type: cross Abstract: We study the problem of sequentially evaluating a new large language model (LLM) on a fixed question set using historical performance data from prior LLMs.

arXiv AI

Jul 1

Test-Time Verification for Text-to-SQL via Outcome Reward Models

arXiv:2606. 30851v1 Announce Type: cross Abstract: Improving the reliability of large language models (LLMs) at inference time is a central challenge in structured reasoning tasks such as Text-to-SQL.

By Mattia Tritto, Giuseppe Farano, Dario Di Palma, Gaetano Rossiello, Fedelucio Narducci, Dharmashankar Subramanian, Tommaso Di Noia

llms benchmarks safety data-engineering

arXiv AI

Aug 5

Interpretable Adaptive Sampling for LLM Test-Time Scaling

arXiv:2608. 03961v1 Announce Type: new Abstract: Test-time scaling improves LLM reasoning by generating and aggregating multiple candidate answers, yet many pipelines use fixed per-query budgets that spend the same compute on easy and difficult prompts.

By Mobina Kashaniyan, Ali Jannesari

llms safety

arXiv AI

Jun 3

Test-Time Optimization of Physical Query Plans with LLMs

arXiv:2602. 10387v2 Announce Type: replace-cross Abstract: Traditional query optimization relies on cost-based optimizers that estimate execution cost (e.

By Mehmet Hamza Erol, Xiangpeng Hao, Federico Bianchi, Ciro Greco, Jacopo Tagliabue, James Zou

llms

arXiv AI

Jul 1

Optimal Self-Consistency for Efficient Reasoning with Large Language Models

arXiv:2511. 12309v2 Announce Type: replace-cross Abstract: Self-consistency (SC) is a widely used test-time inference technique for improving performance in chain-of-thought reasoning.

By Austin Feng, Marius Alonso, Ambroise Odonnat, Vasilii Feofanov, Ievgen Redko

llms benchmarks

arXiv AI

Aug 5

Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility

arXiv:2608. 04001v1 Announce Type: cross Abstract: Large language models can solve substantially harder reasoning problems with more inference-time compute.

By Mohsen Hariri, Weicong Chen, Nahal Shahini, Vikash Singh, Kai Ye, Amirhossein Samandar, Debargha Ganguly, Sreehari Sankar, Yanyan Zhang, Shouren Wang, Jerry Peng, Biyao Zhang, Michael Hinczewski, Vipin Chaudhary

llms benchmarks

arXiv Machine Learning

Jul 2

Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputs

arXiv:2509. 17314v4 Announce Type: replace-cross Abstract: Software increasingly relies on the emergent capabilities of Large Language Models (LLMs), from natural language understanding to program analysis and generation.

By Juyeon Yoon, Somin Kim, Robert Feldt, Shin Yoo

llms benchmarks

arXiv AI

Jun 2

Beyond the Frontier: Stochastic Backtracking for Efficient Test-Time Scaling

arXiv:2605. 25143v2 Announce Type: replace Abstract: Test-time scaling improves language model reasoning by spending additional compute to explore multiple solution trajectories.

By Dao Tran, Duc Anh Le, Ngoc Luu, Quan Pham, Tung Pham, Hung Bui

llms benchmarks

arXiv Machine Learning

Jun 5

Query-efficient model evaluation using cached responses

arXiv:2605. 07096v2 Announce Type: replace Abstract: Evaluating a new model on an existing benchmark is often necessary to understand its behavior before deployment.

By Hayden Helm, Ben Johnson, Carey Priebe

benchmarks

arXiv AI

Aug 6

When Large Language Models Know the Table: A Framework for Assessing Data Contamination in Tabular Datasets

arXiv:2510. 20351v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly exposed to data contamination, i.

By Matteo Silvestri, Fabiano Veglianti, Flavio Giorgi, Fabrizio Silvestri, Gabriele Tolomei

llms

arXiv AI

Jun 11

Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation

arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.

By Selen Erkan, Bastian Boll, Kristian Kersting, Bj\"orn Deiseroth, Letitia Parcalabescu

llms benchmarks

arXiv AI

Jul 14

Fail-Aware and Explainable Test Oracle Prediction

arXiv:2607. 11342v1 Announce Type: cross Abstract: Despite their central role in fault detection, test oracles remain challenging to construct effectively.

By Yue Zhao, Binish Tanveer, Jelena Zdravkovic

llms benchmarks

arXiv AI

Jun 11

TAHOE: Text-to-SQL with Automated Hint Optimization from Experience

arXiv:2606. 12387v1 Announce Type: cross Abstract: Large Language Models (LLMs) have democratized database access through Text-to-SQL, but moving from prototypes to production remains difficult.

By Zhiyi Chen, Jie Song, Peng Li

llms agents fine-tuning data-engineering

Related stories