The paper introduces RIVET, a method for multi‑turn large language model routing that uses post‑call responses to train two value estimators: a deployable Routing Value Estimator that operates on pre‑call information, and a training‑only Response‑Informed Value Estimator that also observes the actual response. By conditioning on the response, RIVET turns response‑induced return variation into an observable covariate, reducing asymptotic estimation error. Experiments on the MINT benchmark show that RIVET solves more held‑out tasks than the strongest baseline while cutting total call cost by 74.7%.
arXiv:2607. 00053v1 Announce Type: cross Abstract: Large language models (LLMs) embedded in multi-turn agentic harnesses are reshaping software engineering (SWE), but routing every task to a frontier model is wasteful when many issues admit cheap fixes.
arXiv:2610.15993v1 Announce Type: new Abstract: We study how initial model selection, later handoff, and training targets affect success and cost in LLM tasks that span multiple turns. We compare sev...
arXiv:2609.37362v1 Announce Type: new Abstract: Large language model (LLM) routing aims to assign each query to the most suitable model from a heterogeneous candidate pool, improving the quality--eff...
arXiv:2605.18859v3 Announce Type: replace-cross Abstract: LLM routing matters most in long-horizon applications such as coding agents, deep research systems, and computer-use agents, where a single u...
The paper introduces RAISE, a diagnostic framework that tests whether a costly large language model (LLM) signal provides enough pre-call information to justify selective use. It identifies the failure mode of acquisition collapse, where an LLM appears useful overall but lacks actionable evidence for individual decisions. The authors demonstrate RAISE with Structured Hypothesis Embeddings (SHE) and evaluate it across multiple study designs, showing that predictable incremental benefit, rather than average lift, indicates recoverable selective value.
arXiv:2609.22951v1 Announce Type: cross Abstract: Enterprise agentic systems that route every trajectory step to a frontier model waste 60-80% of their inference budget on subtasks that smaller model...
arXiv:2608. 15817v1 Announce Type: new Abstract: The growing ecosystem of large language models (LLMs) offers huge potential to optimize performance-cost trade-offs.
arXiv:2607. 22465v1 Announce Type: cross Abstract: Routing to select large language models (LLMs) with different cost-quality trade-offs has become a fundamental deployment feature of enterprise AI.
arXiv:2607. 20481v1 Announce Type: new Abstract: Local-cloud collaboration is a practical way to deploy large language models under resource constraints, but existing methods often rely on trained routers or collaboration-aware finetuning that tie routing behavior to a particular operating regime.
FlexRouter is a routing framework for large language models that explicitly models model complementarity to maximize answer coverage. It formulates routing as a coverage-oriented subset selection problem and uses Determinantal Point Processes to capture both competence and redundancy. During inference, a greedy strategy based on marginal log-determinant gains allows the router to adaptively determine subset sizes without a fixed budget, achieving higher coverage with lower redundancy on the RouterEval benchmark.
arXiv:2606. 19376v1 Announce Type: cross Abstract: Inference costs for large language model (LLM) applications are rapidly growing, driven by surging demand and rising infrastructure cost.
The paper introduces PFN‑OPE, a prior‑data fitted network that performs amortized off‑policy evaluation (OPE) for large language models (LLMs). Unlike traditional OPE methods that must be retrained for each new logged dataset or reward definition, PFN‑OPE is pretrained once on a diverse set of contextual‑bandit tasks and can estimate value for any new logged dataset and target response in a single forward pass. Experiments on HelpSteer2 and UltraFeedback with Qwen, Llama, and Gemma policies show that PFN‑OPE reduces error by 2.0 to 9.3 times compared to the best baselines in reward‑shifted settings.