arXiv AI
Rethinking Quality in Large Language Model Routing: Security-Conditioned Model-and-Budget Selection
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 30
Routing Should Pay for Itself: Sparse Supervision for Economical LLM Routing
The paper introduces SaveRouter, a sparse‑supervision framework for large language model routing that selectively gathers informative model feedback and shares capability information across related queries. By using only about 33–41% of available training feedback, SaveRouter achieves competitive or superior routing quality while reducing the break‑even deployment volume by 1.9–9.5× compared to conventional routers. The study also shows that the supervision level that minimizes serving cost may differ from the one that yields the earliest payback.
Hugging Face Trending Papers
Oct 7
SLDR: Defending Against Malicious Fine-tuning via Selective Layers Recovery and Dynamic Routing
Fine-tuning-as-a-service enables users to adapt aligned large language models (LLMs) to specialized tasks, but malicious fine-tuning can erode refusal behavior while preserving task performance on leg...
arXiv AI
Jun 11
Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models
arXiv:2606. 11409v1 Announce Type: cross Abstract: Adversarial robustness evaluations of large language models (LLMs) typically report attack success rate (ASR) under fixed query budgets, implicitly treating all attacks as equally costly.
arXiv Machine Learning
6d ago
CopRoute: Dependence-Aware Conformal Routing for Language Models
CopRoute introduces a dependence-aware conformal routing method for language models that orders models by the number of new queries each correctly answers beyond those already covered, selecting the minimal cutoff that satisfies a target all‑miss risk. This approach accounts for model failure dependencies—such as shared training data or architecture—that traditional risk‑controlled policies ignore. Experiments on RouterBench and RouterEval show CopRoute achieves fewer certified inference calls than state‑of‑the‑art methods, reducing calls by up to 79 % in some cases while maintaining the same risk level.
arXiv AI
Sep 18
Effective and Efficient Threat Hunting with Small Language Models
The paper presents a framework for translating natural‑language queries into Kusto Query Language (KQL) using small language models (SLMs). It introduces lightweight retrieval, error‑aware prompting, LoRA fine‑tuning with rationale distillation, and a two‑stage architecture that pairs an SLM drafter with a low‑cost LLM judge. Evaluations on Microsoft’s NL2KQL Defender dataset show the two‑stage approach achieving high syntax and schema‑valid accuracy while dramatically reducing cost compared to larger LLM baselines.
arXiv AI
Sep 24
Learning the Cost of Reliable Inference
arXiv:2609.28322v1 Announce Type: new Abstract: Benchmarking and routing platforms increasingly act as intermediaries connecting large language model providers with end-users. However, providers on t...