← Back to all news
arXiv AI By Han Shen, Qifu Tyler Sun, Fanyang Meng, Ye Wang, Yongsheng Liang

PriorityFEC: Deadline-Aware Streaming FEC for KV-Cache Handoff in Edge Multi-Agent Systems

Read the original on arXiv AI →

PriorityFEC is a deadline‑aware streaming forward erasure correction scheme designed to hand off key–value (KV) cache data between agents in edge multi‑agent large language model systems. It prioritizes high‑utility KV blocks in the transmission stream and adapts repair packet scheduling based on receiver feedback to meet strict time‑to‑first‑token deadlines. Experiments with Qwen2.5‑7B‑Instruct and Llama‑3.1‑8B‑Instruct show that PriorityFEC improves task quality compared to other transport schemes, especially under tight deadline constraints.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Aug 10

Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving

arXiv:2608. 06557v1 Announce Type: cross Abstract: The reasoning and agentic capabilities of large language models have expanded the range of applications they support, from short interactive exchanges to long, compute-heavy requests.

By Muhammad Adnan, Rohan Mahapatra, Prashant J. Nair, Daniel Berger, Pantea Zardoshti, Rodrigo Fonseca, Esha Choukse
llmsagentssafety
More like this →
arXiv Computation and Language
Aug 27

TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving

TOPAS is a Task‑Oriented Prefix‑Aware Scheduler designed for multi‑agent large language model serving. It jointly decides which agent prefixes to retain in a shared key‑value cache and which requests to schedule, balancing the reduction of each task’s longest remaining service path against the benefit of downstream prefix reuse while accounting for movement and preemption costs. Experiments on synthetic DAGs and MetaGPT software‑development workflows show that TOPAS can reduce mean and p99 job completion times by up to 39.8%/49.4% and 22.0%/26.6% respectively compared to the best baselines.

By Hongqiu Ni, Han Tian, Chi Zhang, Guopeng Li, Haisheng Tan
llmsagentsefficiency
More like this →