arXiv AI
Do Dynamic Routers Need Memory? HeRo: History-Aware Routing for Efficient LLM Inference
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 30
Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing
arXiv:2609.37362v1 Announce Type: new Abstract: Large language model (LLM) routing aims to assign each query to the most suitable model from a heterogeneous candidate pool, improving the quality--eff...
arXiv Computation and Language
Oct 15
RouteWeaver: Weaving Mode Selection and Execution into Unified LLM Routing
arXiv:2610.19705v1 Announce Type: new Abstract: LLM routing dynamically selects and coordinates heterogeneous language models across diverse tasks. However, existing approaches are largely built arou...
arXiv Machine Learning
Aug 27
Ban&Pick: Enhancing Performance and Efficiency of MoE-LLMs via Smarter Routing
The paper introduces Ban&Pick, a post‑training, plug‑and‑play routing strategy for Sparse Mixture‑of‑Experts large language models. It identifies and reinforces a small group of highly influential experts while dynamically pruning redundant ones, leading to accuracy gains across math, code, and reasoning benchmarks. Experiments on DeepSeek and Qwen3 show notable performance improvements and a 1.25× inference speedup without retraining or architectural changes.
arXiv AI
Aug 7
MACRO: Markov Chain Routing of Transformer Layers
arXiv:2608. 05872v1 Announce Type: cross Abstract: Standard Large Language Models (LLMs) execute layers sequentially.
arXiv Computation and Language
Sep 1
RouteSparse: Input-Conditional Pattern Routing for Budgeted Long-Context Prefilling
arXiv:2608.29058v1 Announce Type: new Abstract: Dynamic sparse attention can reduce the quadratic cost of long-context prefilling without changing model weights. MInference assigns each attention hea...
arXiv Machine Learning
Jul 27
TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI
arXiv:2607. 22465v1 Announce Type: cross Abstract: Routing to select large language models (LLMs) with different cost-quality trade-offs has become a fundamental deployment feature of enterprise AI.