The paper examines how large language model (LLM) routers can overpay by favoring the strongest, most expensive models even when cheaper alternatives offer similar utility, especially under loose cost budgets. This strong-model over-selection is not solely due to ambiguity but arises when cost differences persist despite equivalent performance. To address this, the authors introduce EquiRouter, a cost‑aware routing method that learns query‑dependent rankings and explicitly prefers lower‑cost models when utilities are comparable, achieving better cost efficiency without sacrificing performance.
By Guannan Lai, Long Chen, Han-Jia Ye
arXiv:2605.18859v3 Announce Type: replace-cross
Abstract: LLM routing matters most in long-horizon applications such as coding agents, deep research systems, and computer-use agents, where a single u...
By Pei Yang, Wanyi Chen, Tongyun Yang, Pengbin Feng, Jiarong Xing, Wentao Guo, Yuhang Yao, Yuhang Han, Hanchen Li, Xu Wang, Zeyu Wang, Jie Xiao, Anjie Yang, Liang Tian, Lynn Ai, Eric Yang, Tianyu Shi
arXiv:2606.27457v2 Announce Type: replace-cross
Abstract: Serving open-weight language models on premises forces a trade-off between accuracy and computational cost. Hosting models of different sizes...
By Yasmin Moslem, Magdalena Kacmajor, Vasudevan Nedumpozhimana, Ammar Abbas, Solmaz Panahi, David Lynch, Zhuangzhuang Nie, Alexandros Agapitos, Aleksandar Milenovic, Hongmeng Song, Yucheng Shi, Yue Pan, Patricia Buffini, John D. Kelleher
arXiv:2609.28322v1 Announce Type: new
Abstract: Benchmarking and routing platforms increasingly act as intermediaries connecting large language model providers with end-users. However, providers on t...
By Dimitrios Rontogiannis, Ander Artola Velasco, Manuel Gomez Rodriguez
arXiv:2607. 08665v1 Announce Type: new Abstract: Routing among large language models (LLMs) trades response quality against serving cost, motivated by the reported gap between deployed routers and a per-instance oracle.
By Teng-Ruei Chen
HybridInfer is a thermal‑aware reinforcement‑learning router that selects among on‑device, edge, and cloud large language model tiers based on a phone’s thermal headroom and query complexity. Trained offline with a Q‑learning policy, it balances quality, latency, cost, and a thermal penalty, and includes a locality bonus that encourages on‑device execution. In real Android tests on 210 prompts, the learned router outperformed two hand‑tuned heuristics in quality while maintaining the lowest cost, and it proved more reliable and faster than always‑on‑device inference, especially for long queries.
By Simran Koul