Who Pays for Efficient Inference? Confidence-Based Small-to-Large LLM Routing Across Tasks and Languages
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The Flow has not summarised this story yet — read it at arXiv Computation and Language.
ScoutRoute is a lightweight post‑hoc routing framework for Mixture‑of‑Experts systems that predicts expert competence using internal language model signals from a single forward pass of a scout model. It employs hidden‑state representations with logistic probes and logit‑derived confidence measures in a domain‑aware routing strategy, avoiding modifications to the underlying language models. Experiments on five open‑weight language models across four benchmark datasets show that ScoutRoute consistently outperforms individual routing signals and achieves superior calibration with only one scout‑model inference.
arXiv:2608. 10939v1 Announce Type: cross Abstract: Multilingual short-text classification supports operational systems such as content moderation, customer support routing, and intent recognition, yet aggregate evaluation often hides large differences between high-resource and low-resource languages.
arXiv:2608. 15964v1 Announce Type: cross Abstract: Language-specific competency (LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt.
arXiv:2608. 03803v1 Announce Type: cross Abstract: Multilingual language models are deployed across a hundred or more languages, yet most benchmarks test whether a model can perform a task _in_ a language rather than whether it commands the language itself, conflating fluency with proficiency.
arXiv:2605.28306v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) models enable efficient LLM scaling, yet adapting them to non-English downstream tasks remains challenging. Standard...
arXiv:2607. 25600v1 Announce Type: cross Abstract: Retrieval-augmented generation improves knowledge-intensive question answering, but indiscriminate retrieval can introduce irrelevant evidence and unnecessary computation.