arXiv AI
Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Sep 1
Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey
arXiv:2603.04445v3 Announce Type: replace-cross Abstract: The rapid growth of large language models (LLMs) with diverse capabilities, costs, and domains has created a critical need for intelligent mo...
arXiv AI
3d ago
Routing Should Pay for Itself: Sparse Supervision for Economical LLM Routing
The paper introduces SaveRouter, a sparse‑supervision framework for large language model routing that selectively gathers informative model feedback and shares capability information across related queries. By using only about 33–41% of available training feedback, SaveRouter achieves competitive or superior routing quality while reducing the break‑even deployment volume by 1.9–9.5× compared to conventional routers. The study also shows that the supervision level that minimizes serving cost may differ from the one that yields the earliest payback.
arXiv Machine Learning
Aug 13
LLM Router: Rethinking Routing with Prefill Activations
arXiv:2603. 20895v3 Announce Type: replace-cross Abstract: Existing routers rely on semantic query features or handcrafted features, which often fail to capture model-specific failures or intrinsic task difficulty.
arXiv AI
Sep 10
Do Dynamic Routers Need Memory? HeRo: History-Aware Routing for Efficient LLM Inference
arXiv:2609.08189v1 Announce Type: new Abstract: Dynamic layer routing reduces the inference cost of Large Language Models (LLMs) by learning to skip layers for individual tokens. Existing methods, ho...
arXiv AI
2d ago
FlexRouter: Learning Complementary Model Sets for Flexible LLM Routing
FlexRouter is a routing framework for large language models that explicitly models model complementarity to maximize answer coverage. It formulates routing as a coverage-oriented subset selection problem and uses Determinantal Point Processes to capture both competence and redundancy. During inference, a greedy strategy based on marginal log-determinant gains allows the router to adaptively determine subset sizes without a fixed budget, achieving higher coverage with lower redundancy on the RouterEval benchmark.
arXiv AI
Sep 3
SCX Router: Streaming Zero-Shot Model Selection with a Decoder-KV Classifier and a Real-World Task Ontology
The paper introduces SCX Router, a lightweight GLiClass-based model selector that assigns suitability scores to inference-time language models without autoregressive generation. It uses a 0.6B-parameter Qwen3 decoder with a shallow bidirectional scorer, preserving a text-only key–value cache across sessions and predicting task attributes such as type, difficulty, and expected output length. The authors build a comprehensive task ontology with 23 families, 115 types, and 1,173 synthetic examples, generating 150,000 verifier-scored tasks to train the router, which outperforms baseline models on LiveBench subsets with a top‑1 score of 0.707 versus 0.696 for the strongest fixed model.