arXiv Machine Learning August 11, 2026 By Ibne Farabi Shihab, Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Md Najmus Swaqeeb

Opportunity Is Not Realizability: Selection-Valid Diagnostics for Multi-LLM Routing

Read the original on arXiv Machine Learning →

arXiv:2608. 08265v1 Announce Type: new Abstract: Oracle routing measures how much a pool of language models could gain from per-query selection, but the diagnostic has two flaws: testing against a best fixed model selected on the same examples invalidates paired inference, and a full-information oracle sees outcomes no deployable router observes.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 7

How Much of the Routing Gap Is Real? Decomposing the Router-to-Oracle Gap into Reproducible Specialist Advantage and Single-Draw Label Noise

arXiv:2607. 03436v1 Announce Type: new Abstract: Routing among large language models (LLMs) promises better quality at lower cost, motivated by the reported gap between learned routers and a per-instance oracle.

By Teng-Ruei Chen

llms benchmarks

arXiv AI

1d ago

LLMs Can Predict Failure Risk, But Struggle to Predict Which Collaboration Protocol Pays Off: Cost-Aware Protocol Routing Across Reasoning Tasks

arXiv:2608. 14927v1 Announce Type: new Abstract: Multi-agent large language model (LLM) systems can improve reasoning by spending more computation, but deployment requires deciding when extra collaboration is worth its cost.

By Chih-Hsuan Yang, Jingyan Jiang, Cheng-Hau Yang, Vikram Vasudevan, Huihuo Zheng, Venkatram Vishwanath, Rajeev Thakur

llms agents benchmarks

arXiv AI

Aug 5

Reachability Is Not Realization: Tracing the Sources of LLM Benchmark Gains

arXiv:2608. 03219v1 Announce Type: new Abstract: Benchmark gains are often treated as evidence of greater LLM capability.

By Yanchao Li, Wanhao Liu, Jiaqing Xie, Ben Gao, Yanbo Wang, Tianfan Fu, Yuqiang Li

llms benchmarks

Hugging Face Trending Papers

Aug 4

Reachability Is Not Realization: Tracing the Sources of LLM Benchmark Gains

Benchmark gains are often treated as evidence of greater LLM capability. Yet the same gain can reflect different changes in model behavior.

llms benchmarks

arXiv Machine Learning

Jul 10

Resample or Reroute? Budget-Aware Test-Time Model Selection for Large Language Models

arXiv:2607. 08665v1 Announce Type: new Abstract: Routing among large language models (LLMs) trades response quality against serving cost, motivated by the reported gap between deployed routers and a per-instance oracle.

By Teng-Ruei Chen

llms benchmarks

arXiv Machine Learning

Aug 11

RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enough

arXiv:2608. 07583v1 Announce Type: cross Abstract: Multi-agent LLM systems route among model-backed advisors, yet a deployer rarely knows before shipping whether routing will help at all.

By Anchen Sun, Kaiqi Yang

llms agents benchmarks safety

Related stories

How Much of the Routing Gap Is Real? Decomposing the Router-to-Oracle Gap into Reproducible Specialist Advantage and Single-Draw Label Noise

LLMs Can Predict Failure Risk, But Struggle to Predict Which Collaboration Protocol Pays Off: Cost-Aware Protocol Routing Across Reasoning Tasks

Reachability Is Not Realization: Tracing the Sources of LLM Benchmark Gains

Reachability Is Not Realization: Tracing the Sources of LLM Benchmark Gains

Resample or Reroute? Budget-Aware Test-Time Model Selection for Large Language Models

RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enough