arXiv Machine Learning
Embedding Geometry Shapes LLM Routing: Tracing Failure Modes and Improving Generalization
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Aug 13
LLM Router: Rethinking Routing with Prefill Activations
arXiv:2603. 20895v3 Announce Type: replace-cross Abstract: Existing routers rely on semantic query features or handcrafted features, which often fail to capture model-specific failures or intrinsic task difficulty.
arXiv AI
Jul 15
ReLope: KL-Regularized LoRA Probes for Multimodal LLM Routing
arXiv:2603. 24787v2 Announce Type: replace Abstract: Routing has emerged as a promising strategy for balancing performance and cost in large language model (LLM) systems that combine lightweight models with powerful but expensive large models.
arXiv Computation and Language
Sep 22
Block-Sparse Attention with Semantic-Geometric Decoupled Routing
arXiv:2609.22884v1 Announce Type: new Abstract: Long-context inference has become a defining capability of large language models, but exact dense attention remains costly due to its quadratic scaling...
arXiv AI
Jul 8
Self-Routing: Parameter-Free Expert Routing from Hidden States
arXiv:2604. 00421v2 Announce Type: replace Abstract: Mixture-of-Experts (MoE) layers increase model capacity by activating only a small subset of experts per token, and typically rely on a learned router to map hidden states to expert assignments.
arXiv Machine Learning
Jun 9
The Routing Plateau: Understanding and Breaking the Accuracy Limits of LLM Routers
arXiv:2606. 07587v1 Announce Type: new Abstract: LLM routing has become a popular approach to improve the cost-quality trade-off of LLM services by dynamically selecting a model for each query.
arXiv AI
Oct 9
RouterInterp: Understanding Superposed Specialisation in Mixture of Experts Routing
RouterInterp: Understanding Superposed Specialisation in Mixture of Experts Routing proposes the Superposed Specialisation Hypothesis (SSH), suggesting that experts in Sparse Mixture of Experts models specialise in a disjoint union of fine‑grained features rather than a single domain. The paper introduces RouterInterp, a method that identifies Sparse Autoencoder features predictive of routing decisions and generates unified natural‑language explanations. On the gpt‑oss‑20b model, RouterInterp achieves approximately 65% higher detection accuracy than previous token‑statistics based methods, offering a scalable way to explain expert routing.