arXiv Machine Learning June 5, 2026 By Hayden Helm, Ben Johnson, Carey Priebe

Query-efficient model evaluation using cached responses

Read the original on arXiv Machine Learning →

arXiv:2605. 07096v2 Announce Type: replace Abstract: Evaluating a new model on an existing benchmark is often necessary to understand its behavior before deployment.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

benchmarks

arXiv Machine Learning

Jul 21

Efficient Sequential Evaluation of Large Language Models

arXiv:2607. 17409v1 Announce Type: cross Abstract: We study the problem of sequentially evaluating a new large language model (LLM) on a fixed question set using historical performance data from prior LLMs.

By Chia-Yu Hsu, Shubhanshu Shekhar

llms

arXiv Machine Learning

Jun 19

Closing the Calibration Gap in Semantic Caching

arXiv:2606. 19719v1 Announce Type: cross Abstract: Semantic caching cuts LLM inference costs by serving a cached response to semantically similar queries.

By Aditeya Baral, Radoslav Ralev, Iliya Sotirov Zhechev, Srijith Rajamohan, Jen Agarwal

llms

arXiv AI

Jun 26

The Capability Frontier: Benchmarks Miss 82% of Model Performance

arXiv:2606. 26836v1 Announce Type: new Abstract: Existing benchmarks typically report accuracy for a single model on a single run.

By Bradley Fowler, Ryan Smith, Daniel Thi Graviet, William Myers, Joshua Greaves, Narmeen Fatimah Oozeer, Ant\'ia Garc\'ia, Philip Quirke, Amirali Abdullah, Fazl Barez, Shriyash Kaustubh Upadhyay

llms agents benchmarks

arXiv AI

Jun 3

Cost-Aware Query Routing in RAG: Empirical Analysis of Retrieval Depth Tradeoffs

arXiv:2606. 02581v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) faces a fundamental three-way tension: deeper retrieval improves factual grounding but inflates token costs and end-to-end latency.

By Sanjay Mishra

llms rag benchmarks safety

arXiv AI

Jul 7

Risk-Constrained Freshness-Aware Semantic Caching for Open-Web Retrieval-Augmented LLMs

arXiv:2607. 04281v1 Announce Type: cross Abstract: Semantic caching reduces the latency and cost of retrieval-augmented generation (RAG) by serving cached answers to semantically similar queries, but most existing methods do not model the time-varying freshness of open-web evidence.

By Muhammad Mansoor, Tahir Ahmad, Yeo-Chan Yoon

llms rag benchmarks

arXiv AI

Aug 5

Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls

arXiv:2608. 03071v1 Announce Type: new Abstract: Large language model agents derive much of their capability from tool use.

By Guoyao Yu, Xiaoqing Sun, Ziqi Huang, Shaojing Fan, Zhongyi Zhang, Xiaomeng Hu, Xiaobo Xue, Yangyang Shi, Xiong Xiao, Yang Song, Biao Lyu, Rong Wen, Xing Li, Qinming He, Shunming Zhu, Zhenguang Liu

llms rag agents fine-tuning benchmarks

Related stories

Efficient Sequential Evaluation of Large Language Models

Closing the Calibration Gap in Semantic Caching

The Capability Frontier: Benchmarks Miss 82% of Model Performance

Cost-Aware Query Routing in RAG: Empirical Analysis of Retrieval Depth Tradeoffs

Risk-Constrained Freshness-Aware Semantic Caching for Open-Web Retrieval-Augmented LLMs

Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls