arXiv AI
Do Dynamic Routers Need Memory? HeRo: History-Aware Routing for Efficient LLM Inference
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Aug 27
Ban&Pick: Enhancing Performance and Efficiency of MoE-LLMs via Smarter Routing
The paper introduces Ban&Pick, a post‑training, plug‑and‑play routing strategy for Sparse Mixture‑of‑Experts large language models. It identifies and reinforces a small group of highly influential experts while dynamically pruning redundant ones, leading to accuracy gains across math, code, and reasoning benchmarks. Experiments on DeepSeek and Qwen3 show notable performance improvements and a 1.25× inference speedup without retraining or architectural changes.
arXiv AI
Aug 7
MACRO: Markov Chain Routing of Transformer Layers
arXiv:2608. 05872v1 Announce Type: cross Abstract: Standard Large Language Models (LLMs) execute layers sequentially.
arXiv Computation and Language
Sep 1
RouteSparse: Input-Conditional Pattern Routing for Budgeted Long-Context Prefilling
arXiv:2608.29058v1 Announce Type: new Abstract: Dynamic sparse attention can reduce the quadratic cost of long-context prefilling without changing model weights. MInference assigns each attention hea...
arXiv Machine Learning
Jul 27
TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI
arXiv:2607. 22465v1 Announce Type: cross Abstract: Routing to select large language models (LLMs) with different cost-quality trade-offs has become a fundamental deployment feature of enterprise AI.
arXiv Machine Learning
Aug 12
Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation
arXiv:2608. 10392v1 Announce Type: new Abstract: Mixture-of-experts (MoE) models have recently moved beyond routing a fixed number of complete experts.
arXiv AI
Jun 9
WhiFlash: Accelerating Speculative Decoding with Token-Level Cross-Paradigm Routing
arXiv:2606. 07710v1 Announce Type: cross Abstract: The autoregressive nature of large language models (LLMs) remains a significant bottleneck for inference, particularly in complex agentic workloads.