arXiv AI
End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Aug 27
Multi-Turn Reasoning LLMs for Task Offloading in Mobile Edge Computing
The paper introduces COMLLM, a generative framework that combines Group Relative Policy Optimization with a Look‑Ahead Collaborative Simulation to enable multi‑turn reasoning for task offloading in Mobile Edge Computing. By performing multi‑step Monte Carlo rollouts that jointly model server queue dynamics, COMLLM incorporates long‑term system evolution into its reward design, achieving near‑optimal latency and improved load‑balancing fairness. The framework demonstrates zero‑shot scalability to larger network topologies, outperforming supervised fine‑tuning, deep reinforcement learning, and heuristic baselines without requiring retraining.
arXiv Machine Learning
Aug 10
Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving
arXiv:2608. 06557v1 Announce Type: cross Abstract: The reasoning and agentic capabilities of large language models have expanded the range of applications they support, from short interactive exchanges to long, compute-heavy requests.
arXiv AI
Aug 7
BALANCE: Hybrid Autoregressive-Speculative LLM Inference in Wireless Edge Networks
arXiv:2608. 05926v1 Announce Type: cross Abstract: Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks.
arXiv Machine Learning
Aug 31
DART-FL: Burst-Aware Multitask Federated Learning under Dynamic Inference Demand at the Edge
DART-FL is a multitask federated learning framework designed for edge devices that must balance online inference and model training under limited resources. It dynamically allocates resources between inference and training based on current inference backlog and service capacity, then distributes remaining training capacity among tasks using a queue‑aware scheduler that adjusts loss weights. Experiments on image classification datasets with synthetic and real workloads show that DART‑FL adapts to bursty inference demand, improving accuracy for high‑demand tasks while preserving overall multitask performance.
arXiv AI
Jun 30
LLM Serving Optimization with Variable Prefill and Decode Lengths
arXiv:2508. 06133v4 Announce Type: replace-cross Abstract: We study offline scheduling for large language model (LLM) serving under a fixed KV-cache memory budget, where requests have heterogeneous prompt (prefill) and response (decode) lengths.
arXiv AI
Aug 7
Multi-Agent Transformer for Queue-Level XR Traffic Scheduling in TSN Networks
arXiv:2608. 05340v1 Announce Type: cross Abstract: Time-Sensitive Networking (TSN) and Mobile Edge Computing (MEC) hold strong potential for enabling ultra-reliable low-latency communication for time-sensitive applications, such as eXtended Reality (XR).