arXiv Machine Learning

Learning-Based Collaborative MEC for LLM Inference with Soft-Deadline Awareness via Transformer-Enhanced PPO

arXiv:2608. 02031v1 Announce Type: cross Abstract: This paper investigates collaborative mobile edge computing (MEC) servers for large language model (LLM) inference under soft deadline constraints.

arXiv AI

Aug 7

BALANCE: Hybrid Autoregressive-Speculative LLM Inference in Wireless Edge Networks

arXiv:2608. 05926v1 Announce Type: cross Abstract: Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks.

By Guanqiao Qu, Shuo Chen, Qian Chen, Kin K. Leung, Xianhao Chen

llms efficiency

Hugging Face Trending Papers

Aug 6

BALANCE: Hybrid Autoregressive-Speculative LLM Inference in Wireless Edge Networks

Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks. LLM inference mainly relies on two approaches: Autoregressive decoding (AD) generates output tokens sequentially, resulting in long latency; Speculative decoding (SD) accelerates inference by using a small language model (SLM) to generate multiple draft tokens for LLM verification, but incurs extra memory costs.

llms efficiency

arXiv Machine Learning

Jul 14

Device-Cloud Collaborative LLM Inference with Multi-Modal, Multi-Task, Multi-Turn Conversations

arXiv:2502. 11007v5 Announce Type: replace Abstract: Compared to traditional machine learning models, recent large language models (LLMs) can exhibit multi-task-solving capabilities through multi-modal data sources and multi-turn conversations.

By Liangqi Yuan, Dong-Jun Han, Shiqiang Wang, Christopher G. Brinton

llms reinforcement-learning multimodal

arXiv AI

Jun 2

Deft Scheduling of Dynamic Cloud Workflows with Varying Deadlines via Mixture-of-Experts

arXiv:2606. 01162v1 Announce Type: new Abstract: Workflow scheduling in cloud computing demands the intelligent allocation of dynamically arriving, graph-structured workflows with varying deadlines onto ever-changing virtual machine resources.

By Ya Shen, Gang Chen, Hui Ma, Mengjie Zhang

reinforcement-learning benchmarks

arXiv AI

Jun 11

Resource-Aware LLM Reasoning for Mobile Edge General Intelligence

arXiv:2509. 23248v3 Announce Type: replace Abstract: The rapid advancement of large language models (LLMs) has enabled an emergence of agentic artificial intelligence (AI) with powerful reasoning and autonomous decision-making capabilities.

By Mingyi Luo, Ruichen Zhang, Xiangwang Hou, Jun Du, Chunxiao Jiang, Yong Ren, Shiwen Mao

llms agents safety

arXiv AI

Aug 7

Multi-Agent Transformer for Queue-Level XR Traffic Scheduling in TSN Networks

arXiv:2608. 05340v1 Announce Type: cross Abstract: Time-Sensitive Networking (TSN) and Mobile Edge Computing (MEC) hold strong potential for enabling ultra-reliable low-latency communication for time-sensitive applications, such as eXtended Reality (XR).

By Marcos Carvalho, Fatih Temiz, Shavbo Salehi, Melike Erol-Kantarci, Daniel F. Macedo

llms agents reinforcement-learning

arXiv Machine Learning

Jun 4

Efficient Reasoning on the Edge

arXiv:2603. 16867v2 Announce Type: replace Abstract: Large language models (LLMs) with chain-of-thought reasoning achieve state-of-the-art performance across complex problem-solving tasks, but their verbose reasoning traces and large context requirements make them impractical for edge deployment.

By Yelysei Bondarenko, Thomas Hehn, Rob Hesselink, Romain Lepert, Fabio Valerio Massoli, Evgeny Mironov, Leyla Mirvakhabova, Tribhuvanesh Orekondy, Spyridon Stasis, Andrey Kuzmin, Anna Kuzina, Markus Nagel, Ankita Nayak, Corrado Rainone, Ork de Rooij, Paul N Whatmough, Arash Behboodi, Babak Ehteshami Bejnordi

llms reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Jun 29

Ranking Before Serving: Low-Latency LLM Serving via Pairwise Learning-to-Rank

arXiv:2510. 03243v3 Announce Type: replace-cross Abstract: Efficient scheduling of large language model (LLM) inference tasks is critical for achieving low latency and high throughput, a challenge that is becoming increasingly acute with the rise of reasoning-capable LLMs whose generation lengths are highly variable.

By Yiheng Tao, Yihe Zhang, Matthew Dearing, Xin Wang, Yuping Fan, Michael E. Papka, Zhiling Lan

llms efficiency benchmarks

arXiv AI

Jul 28

Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization

arXiv:2607. 22583v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved widespread adoption because of their strong reasoning and query-response capabilities.

By Muhammad Junaid Ali, Smail Niar, El-Ghazali Talbi

llms efficiency

arXiv Machine Learning

Jun 17

OmniPlan: An Adaptive Framework for Timely and Near-Optimal Network Planning Optimization

arXiv:2606. 18105v1 Announce Type: cross Abstract: Network planning optimization is a fundamental problem across diverse domains, including transportation systems, communication networks, and power grids.

By Longlong Zhu, Jiashuo Yu, Zedi Chen, Yuhan Wu, Zhifan Jiang, Yuchen Xian, Yimeng Liu, Jiajie Su, Shaopeng Zhou, Xingyuan Li, Hongyan Liu, Xuan Liu, Dong Zhang, Chunming Wu, Xiang Chen

llms reinforcement-learning

arXiv AI

6d ago

User-Assisted Collaborative Distributed Inference for Efficient QoS-Aware Autoscaling

arXiv:2608. 11840v1 Announce Type: cross Abstract: Growing demand for artificial intelligence (AI) inference services requires scalable infrastructure, yet centralized serving costs rise with demand.

By Alfreds Lapkovskis, Ali Beikmohammadi, Sindri Magn\'usson, Praveen Kumar Donta

arXiv Machine Learning

1d ago

Beyond Binary Priorities: Multi-Tier SLA Scheduling for Large Language Model Serving

arXiv:2608. 16336v1 Announce Type: cross Abstract: Modern LLM serving deployments must simultaneously satisfy heterogeneous service-level objectives (SLOs) across a diverse population of user tiers, ranging from latency-critical API calls to background batch processing.

By Anders Vestrum, Arya Raeesi, Hanna Roed

llms

Related stories