arXiv:2606. 19376v1 Announce Type: cross Abstract: Inference costs for large language model (LLM) applications are rapidly growing, driven by surging demand and rising infrastructure cost.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
The paper introduces SaveRouter, a sparse‑supervision framework for large language model routing that selectively gathers informative model feedback and shares capability information across related queries. By using only about 33–41% of available training feedback, SaveRouter achieves competitive or superior routing quality while reducing the break‑even deployment volume by 1.9–9.5× compared to conventional routers. The study also shows that the supervision level that minimizes serving cost may differ from the one that yields the earliest payback.
arXiv:2607. 03948v1 Announce Type: new Abstract: We study the online routing problem in large language model serving, where requests arrive sequentially and must be dispatched to parallel decode workers under tight batch-size and KV-cache constraints.
arXiv:2609.28322v1 Announce Type: new Abstract: Benchmarking and routing platforms increasingly act as intermediaries connecting large language model providers with end-users. However, providers on t...
arXiv:2609.23085v1 Announce Type: cross Abstract: Large language models (LLMs) and agentic AI systems are creating rapidly growing inference energy demands as model sizes grow and reasoning trajector...
arXiv:2607. 22465v1 Announce Type: cross Abstract: Routing to select large language models (LLMs) with different cost-quality trade-offs has become a fundamental deployment feature of enterprise AI.
arXiv:2603.04445v3 Announce Type: replace-cross Abstract: The rapid growth of large language models (LLMs) with diverse capabilities, costs, and domains has created a critical need for intelligent mo...