The paper introduces a unified framework called cascade routing that combines routing and cascading strategies for selecting large language models (LLMs) in agentic systems. It derives an optimal cascading strategy, proves the optimality of an existing routing strategy, and demonstrates that integrating both leads to a theoretically optimal approach. Experiments show cascade routing consistently outperforms the individual methods, and the study identifies high‑quality estimators as crucial for effective model selection.
arXiv:2603.04445v3 Announce Type: replace-cross Abstract: The rapid growth of large language models (LLMs) with diverse capabilities, costs, and domains has created a critical need for intelligent mo...
The paper investigates how performance predictors can improve model routing for large language models (LLMs). It shows that a predictor only outperforms a model’s self‑confidence when the candidate model is suboptimal or the predictor adds significant capacity. The authors provide theoretical bounds, extend the analysis to two‑model cascades, and confirm that predictors help most with suboptimal models, with gains diminishing after fine‑tuning.
arXiv:2607. 11399v1 Announce Type: cross Abstract: Large language model agents are increasingly executed not by a single model call, but by an execution harness that manages observation, context, control, action, state, and verification.
arXiv:2608. 15817v1 Announce Type: new Abstract: The growing ecosystem of large language models (LLMs) offers huge potential to optimize performance-cost trade-offs.
arXiv:2606. 22902v3 Announce Type: replace Abstract: Real-world users typically have access to multiple Large Language Models (LLMs) from different providers, and these LLMs often excel at distinct domains, yet none dominate all.
The paper introduces SaveRouter, a sparse‑supervision framework for large language model routing that selectively gathers informative model feedback and shares capability information across related queries. By using only about 33–41% of available training feedback, SaveRouter achieves competitive or superior routing quality while reducing the break‑even deployment volume by 1.9–9.5× compared to conventional routers. The study also shows that the supervision level that minimizes serving cost may differ from the one that yields the earliest payback.
The paper investigates how performance predictors can improve model routing for large language models (LLMs). It shows that a predictor only outperforms a model’s self‑confidence when the candidate model is suboptimal or the predictor adds significant capacity. The authors provide theoretical bounds, extend the analysis to two‑model cascades, and confirm their findings experimentally across 16 LLMs on four benchmarks, noting that the advantage diminishes as models are fine‑tuned.
arXiv:2609.37362v1 Announce Type: new Abstract: Large language model (LLM) routing aims to assign each query to the most suitable model from a heterogeneous candidate pool, improving the quality--eff...
arXiv:2606. 19376v1 Announce Type: cross Abstract: Inference costs for large language model (LLM) applications are rapidly growing, driven by surging demand and rising infrastructure cost.
arXiv:2606.27457v2 Announce Type: replace-cross Abstract: Serving open-weight language models on premises forces a trade-off between accuracy and computational cost. Hosting models of different sizes...
arXiv:2606. 28925v1 Announce Type: cross Abstract: Tool and agent routing from natural-language prompts is naturally a set-valued prediction problem: a single query may require multiple agents, while over-selection increases execution cost.
COMPASS is an agentic router that uses a capability-oriented model profiling approach to select large language models (LLMs) efficiently. It follows a routing recipe, gathers performance and cost evidence through tools, and can inspect candidate answers before making a selection. In evaluations on LLMRouterBench and DeepSWE, COMPASS improves accuracy while reducing model‑call cost compared to specialized and trained routing baselines, and it can adapt to new models without retraining.