arXiv AI
Jumping the Line: Exploiting Length Predictions in LLM Scheduling
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 29
Ranking Before Serving: Low-Latency LLM Serving via Pairwise Learning-to-Rank
arXiv:2510. 03243v3 Announce Type: replace-cross Abstract: Efficient scheduling of large language model (LLM) inference tasks is critical for achieving low latency and high throughput, a challenge that is becoming increasingly acute with the rise of reasoning-capable LLMs whose generation lengths are highly variable.
arXiv Machine Learning
Jun 18
Beyond Prediction: Tail-Aware Scheduling for LLM Inference
arXiv:2606. 18431v1 Announce Type: new Abstract: LLM serving exhibits extreme length variability, making size-based scheduling difficult in practice.
arXiv Computation and Language
Sep 2
OUTLETS: Output-Length Prediction from Speculative Decoding Backbones
arXiv:2609.01068v1 Announce Type: new Abstract: The heavy-tailed distribution of output lengths in Large Language Model (LLM) serving poses major challenges for resource provisioning and cluster sche...
arXiv AI
Jun 30
LLM Serving Optimization with Variable Prefill and Decode Lengths
arXiv:2508. 06133v4 Announce Type: replace-cross Abstract: We study offline scheduling for large language model (LLM) serving under a fixed KV-cache memory budget, where requests have heterogeneous prompt (prefill) and response (decode) lengths.
arXiv AI
Sep 1
Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuning
The paper introduces FAB, an attack that uses meta‑learning to embed dormant adversarial behaviors into large language models (LLMs). These behaviors remain inactive until the model is finetuned by downstream users, at which point the model can exhibit unwanted actions such as unsolicited advertising, jailbreakability, or over‑refusal. FAB is shown to be effective across multiple LLMs and resilient to various finetuning settings.
arXiv AI
Jul 23
FineServe: A Fine-Grained Dataset and Characterization of Global LLM Serving Workloads
arXiv:2607. 19349v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as always-on online services, making efficient LLM serving a critical systems challenge.