arXiv AI

User-Assisted Collaborative Distributed Inference for Efficient QoS-Aware Autoscaling

arXiv:2608. 11840v1 Announce Type: cross Abstract: Growing demand for artificial intelligence (AI) inference services requires scalable infrastructure, yet centralized serving costs rise with demand.

arXiv AI

Jun 3

Brief Announcement: Generative Markov Model for Distributed Computing Systems

arXiv:2606. 03061v1 Announce Type: cross Abstract: Emerging distributed computing paradigms, such as the computing continuum, are inherently heterogeneous, stochastic, and complex.

By Alfreds Lapkovskis, Ali Beikmohammadi, Sindri Magn\'usson, Praveen Kumar Donta

reinforcement-learning

arXiv Machine Learning

Jul 30

Harnessing Large Language Models for Intelligent Resource Allocation in the Internet of Everything

arXiv:2607. 26602v1 Announce Type: cross Abstract: The rapid development of the Internet of Everything (IoE) is accelerating the adoption of intelligent applications.

By Haijun Zhang, Zhuojun Duan, Zijun Wu, Xu Ma, Yuzheng Ren

llms

arXiv Machine Learning

Jul 22

Multi-Timescale Latent-Action DRL for Joint Optimization in Edge-Cloud Networks

arXiv:2607. 18288v1 Announce Type: new Abstract: Load imbalance across edge and cloud layers degrades latency performance in hierarchical edge-cloud computing (HECC) systems under dynamic task arrivals and heterogeneous resources, leading to severe queuing delays and inefficient resource utilization.

By Vo Phi Son, Van-Dinh Nguyen, Ngoc Hung Nguyen, Trinh Van Chien, Symeon Chatzinotas

reinforcement-learning

arXiv AI

Jun 11

INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration

arXiv:2606. 11440v1 Announce Type: new Abstract: Existing multi-agent LLM orchestration methods, ranging from brute-force ensembles to learned routers, select models and topologies based on task and model features.

By Ahasan Kabir, Jiaqi Xue, Mengxin Zheng, Qian Lou

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Aug 4

Learning-Based Collaborative MEC for LLM Inference with Soft-Deadline Awareness via Transformer-Enhanced PPO

arXiv:2608. 02031v1 Announce Type: cross Abstract: This paper investigates collaborative mobile edge computing (MEC) servers for large language model (LLM) inference under soft deadline constraints.

By Ngoc Hung Nguyen, Bjorn Landfeldt

llms reinforcement-learning

arXiv Machine Learning

Jul 8

Joint Energy Management and Coordinated AIGC Workload Scheduling for Distributed Data Centers: A Diffusion-Aided Reward Shaping Approach

arXiv:2605. 02965v2 Announce Type: replace Abstract: Artificial intelligence-generated content (AIGC) has emerged as a transformative paradigm for automating the creation of diverse and customized content, giving rise to rapidly growing computational workloads in cloud data centers.

By Yang Fu, Peng Qin, Liming Chen, Zihao Zhang, Hao Yu, Yifei Wang

diffusion reinforcement-learning efficiency benchmarks

arXiv AI

Jun 30

LLM Serving Optimization with Variable Prefill and Decode Lengths

arXiv:2508. 06133v4 Announce Type: replace-cross Abstract: We study offline scheduling for large language model (LLM) serving under a fixed KV-cache memory budget, where requests have heterogeneous prompt (prefill) and response (decode) lengths.

By Meixuan Wang, Yinyu Ye, Zijie Zhou

llms nlp

arXiv AI

Jun 30

SwarmX: Agentic Scheduling for Low-Latency Agentic Systems

arXiv:2606. 21401v2 Announce Type: replace-cross Abstract: Agentic AI applications compose multiple model calls and tool executions, creating new scheduling challenges for GPU-CPU clusters.

By Yeqi Huang, Yanwei Ye, Guomin Chen, Wenhao Su, Bin Gong, Jialian Li, Zhan Lu, Yangshen Deng, Xuan Sun, Le Xu, Luo Mai

agents efficiency multimodal benchmarks

arXiv Machine Learning

Jun 3

DriftSched: Adaptive QoS-Aware Scheduling under Runtime Token Drift for Multi-Tenant GPU Inference

arXiv:2606. 02982v1 Announce Type: cross Abstract: The rapid growth of large language model (LLM) inference services has increased the demand for efficient multi-tenant GPU scheduling.

By Kathiravan Palaniappan

llms efficiency benchmarks safety

arXiv Machine Learning

Jul 14

Device-Cloud Collaborative LLM Inference with Multi-Modal, Multi-Task, Multi-Turn Conversations

arXiv:2502. 11007v5 Announce Type: replace Abstract: Compared to traditional machine learning models, recent large language models (LLMs) can exhibit multi-task-solving capabilities through multi-modal data sources and multi-turn conversations.

By Liangqi Yuan, Dong-Jun Han, Shiqiang Wang, Christopher G. Brinton

llms reinforcement-learning multimodal

arXiv Machine Learning

Jun 9

BlendServe: Optimizing Offline Inference for Auto-regressive Large Models with Resource-aware Batching

arXiv:2411. 16102v2 Announce Type: replace Abstract: Offline batch inference, which leverages the flexibility of request batching to achieve higher throughput and lower costs, is becoming more popular for latency-insensitive applications.

By Yilong Zhao, Shuo Yang, Kan Zhu, Lianmin Zheng, Baris Kasikci, Yang Zhou, Jiarong Xing, Ion Stoica

efficiency multimodal

arXiv AI

Aug 7

BALANCE: Hybrid Autoregressive-Speculative LLM Inference in Wireless Edge Networks

arXiv:2608. 05926v1 Announce Type: cross Abstract: Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks.

By Guanqiao Qu, Shuo Chen, Qian Chen, Kin K. Leung, Xianhao Chen

llms efficiency

Related stories