The paper introduces a unified framework called cascade routing that combines routing and cascading strategies for selecting large language models (LLMs) in agentic systems. It derives an optimal cascading strategy, proves the optimality of an existing routing strategy, and demonstrates that integrating both leads to a theoretically optimal approach. Experiments show cascade routing consistently outperforms the individual methods, and the study identifies high‑quality estimators as crucial for effective model selection.
By Jasper Dekoninck, Maximilian Baader, Martin Vechev
arXiv:2609.37362v1 Announce Type: new
Abstract: Large language model (LLM) routing aims to assign each query to the most suitable model from a heterogeneous candidate pool, improving the quality--eff...
By Guannan Lai, Han-Jia Ye
arXiv:2609.23085v1 Announce Type: cross
Abstract: Large language models (LLMs) and agentic AI systems are creating rapidly growing inference energy demands as model sizes grow and reasoning trajector...
By Muhammad Abdur Rab Siddiqui, Daniela Rojas, Chen Yang, Wenqi Cui, Yuanyuan Shi, Yize Chen
arXiv:2511. 09373v2 Announce Type: replace-cross Abstract: LLMs now tackle a wide range of software-related tasks, yet we show that their performance varies markedly both across and within these tasks.
By Adam \v{S}torek, Vikas Upadhyay, Marianne Menglin Liu, Daniel W. Peterson, Anshul Mittal, Sujeeth Bharadwaj, Fahad Shah, Sujith Ravi, Dan Roth
arXiv:2602.03478v2 Announce Type: replace
Abstract: LLM routing aims to reduce inference costs by assigning easy queries to less expensive models and reserving stronger models for cases in which they...
By Guannan Lai, Long Chen, Han-Jia Ye
The paper introduces VDAR-Router, a routing framework for large language models that uses verbalized query difficulty analysis to guide model selection. It first generates an explicit difficulty profile for each query, retrieves historical examples with similar profiles, and then estimates model suitability to choose a model based on a reward function balancing performance and cost. Experiments on three datasets show that VDAR-Router consistently outperforms existing baselines in cost‑performance trade‑offs, and case studies confirm that explicit difficulty analysis improves example relevance and routing reliability.
By Yu-Chien Tang, Jun-Chen Hung, Wen-Chih Peng, An-Zi Yen
arXiv:2606. 07587v1 Announce Type: new Abstract: LLM routing has become a popular approach to improve the cost-quality trade-off of LLM services by dynamically selecting a model for each query.
By Yifan Lu, Qiyue Zhang, Shenrun Zhang, Zhibo Yu, Zhuang Wang, Hanjie Chen, Jiarong Xing
The paper introduces SaveRouter, a sparse‑supervision framework for large language model routing that selectively gathers informative model feedback and shares capability information across related queries. By using only about 33–41% of available training feedback, SaveRouter achieves competitive or superior routing quality while reducing the break‑even deployment volume by 1.9–9.5× compared to conventional routers. The study also shows that the supervision level that minimizes serving cost may differ from the one that yields the earliest payback.
By Guannan Lai, Gelin Bian, Hao-Xuan Ma, Jun-Peng Jiang, Long Chen, Jian-Dong Liu, Zhi-Hao Tan, Han-Jia Ye
arXiv:2610.19705v1 Announce Type: new
Abstract: LLM routing dynamically selects and coordinates heterogeneous language models across diverse tasks. However, existing approaches are largely built arou...
By Xiaohan Wang, Haozhen Zhang, Qingyuan Liu, Tao Feng, Wenya Wang
FlexRouter is a routing framework for large language models that explicitly models model complementarity to maximize answer coverage. It formulates routing as a coverage-oriented subset selection problem and uses Determinantal Point Processes to capture both competence and redundancy. During inference, a greedy strategy based on marginal log-determinant gains allows the router to adaptively determine subset sizes without a fixed budget, achieving higher coverage with lower redundancy on the RouterEval benchmark.
By Wang Wei, Harry Yang, Tiankai Yang, Samyadeep Basu, Hongjie Chen, Andy Zhao, Franck Dernoncourt, Ryan A. Rossi, Hoda Eldardiry
The paper introduces EDRM, an entropy‑dynamics‑based routing framework that decides whether to use direct, standard, or chain‑of‑thought decoding for large language models. By analyzing early decoding entropy, EDRM identifies tasks that benefit from explicit reasoning and routes queries accordingly, achieving 16–55% token savings while maintaining accuracy across 15 benchmarks and 7 LLMs. The study also shows that a short direct fallback branch can further improve accuracy by up to 4.9 percentage points compared to routing alone.
By Wei Xia, Haoqing Wang, Zhi-Hong Deng, Yehui Tang
The paper investigates how performance predictors can improve model routing for large language models (LLMs). It shows that a predictor only outperforms a model’s self‑confidence when the candidate model is suboptimal or the predictor adds significant capacity. The authors provide theoretical bounds, extend the analysis to two‑model cascades, and confirm that predictors help most with suboptimal models, with gains diminishing after fine‑tuning.