Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks. LLM inference mainly relies on two approaches: Autoregressive decoding (AD) generates output tokens sequentially, resulting in long latency; Speculative decoding (SD) accelerates inference by using a small language model (SLM) to generate multiple draft tokens for LLM verification, but incurs extra memory costs.
arXiv:2608.05926v2 Announce Type: replace-cross
Abstract: Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks. LLM infer...
By Guanqiao Qu, Shuo Chen, Qian Chen, Kin K. Leung, Xianhao Chen
S2-MoE is a self‑speculative decoding framework designed to make Mixture‑of‑Experts (MoE) inference more efficient on edge devices. It reduces verification overhead by using routing‑aware adaptive speculative expansion, improves verification efficiency with reuse‑aware expert gating, and aligns draft and target execution through shared context. Implemented in llama.cpp, S2‑MoE delivers up to 5.3× speedup (≈2.0× on average) over standard autoregressive decoding across various MoE models and datasets on edge hardware.
By Haochen Huang, Shengxuan Qiu, Meng Li
arXiv:2609.17193v1 Announce Type: new
Abstract: Large language model (LLM)-powered agentic AI services increasingly demand low-latency inference, motivating the deployment of LLMs across distributed...
By Zhen Li, Jun Cai, Haoran Gao, An Li, Tan Li
The paper introduces COMLLM, a generative framework that combines Group Relative Policy Optimization with a Look‑Ahead Collaborative Simulation to enable multi‑turn reasoning for task offloading in Mobile Edge Computing. By performing multi‑step Monte Carlo rollouts that jointly model server queue dynamics, COMLLM incorporates long‑term system evolution into its reward design, achieving near‑optimal latency and improved load‑balancing fairness. The framework demonstrates zero‑shot scalability to larger network topologies, outperforming supervised fine‑tuning, deep reinforcement learning, and heuristic baselines without requiring retraining.
By Ning Yang, Chuangxin Cheng, Haijun Zhang
arXiv:2608. 02031v1 Announce Type: cross Abstract: This paper investigates collaborative mobile edge computing (MEC) servers for large language model (LLM) inference under soft deadline constraints.
By Ngoc Hung Nguyen, Bjorn Landfeldt
Diffusion language models (DLMs) provide a non‑autoregressive approach for mobile edge agentic AI, refining tokens through iterative denoising instead of left‑to‑right decoding. They can update multiple uncertain tokens in parallel and use bidirectional context, allowing flexible quality‑latency trade‑offs and early exits that reduce response delay and communication overhead. The survey reviews DLM foundations, resource‑efficient architectures, training and inference acceleration, compression, deployment strategies, and discusses open issues such as long‑context management, split inference, and trustworthy execution.
By Chenqi Li, Minghui Min, Dusit Niyato, Wei Ni
arXiv:2608. 13076v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable success in natural language understanding and generation, but their deployment is constrained by high computational demands.
By Divya Jyoti Bajpai, Kishan Kumar Upadhyay, Manjesh Kumar Hanawal
arXiv:2603. 16867v2 Announce Type: replace Abstract: Large language models (LLMs) with chain-of-thought reasoning achieve state-of-the-art performance across complex problem-solving tasks, but their verbose reasoning traces and large context requirements make them impractical for edge deployment.
By Yelysei Bondarenko, Thomas Hehn, Rob Hesselink, Romain Lepert, Fabio Valerio Massoli, Evgeny Mironov, Leyla Mirvakhabova, Tribhuvanesh Orekondy, Spyridon Stasis, Andrey Kuzmin, Anna Kuzina, Markus Nagel, Ankita Nayak, Corrado Rainone, Ork de Rooij, Paul N Whatmough, Arash Behboodi, Babak Ehteshami Bejnordi
arXiv:2606. 03770v1 Announce Type: cross Abstract: Large Language Models (LLMs) have become integral to modern applications, yet their deployment remains challenging.
By Truong-Thanh Le, Amir Taherkordi, Hoang-Loc La, Frank Eliassen, Phuong Hoai Ha, Peiyuan Guan
arXiv:2604. 26508v2 Announce Type: replace-cross Abstract: Deploying Vision-Language Models (VLMs) on edge devices remains challenging due to their substantial computational and memory demands, which exceed the capabilities of resource-constrained embedded platforms.
By Cyril Shih-Huan Hsu, Wig Yuan-Cheng Cheng, Chrysa Papagianni
arXiv:2608.28726v1 Announce Type: new
Abstract: The remarkable performance of multimodal large language models (MLLMs) comes at the cost of substantial computational overhead, posing significant chal...
By Xinyuan Gui, Shaowen Wang, Sheng Sun, Zijian Wang, Zishu Yu, Zheming Yang