arXiv AI

CANS: Accelerating Multiuser Collaborative Edge Inference via Cooperative Autodidactic NeuroSurgeon

arXiv:2606. 09175v1 Announce Type: cross Abstract: Recently, mobile edge computing (MEC)-enabled collaborative deep neural network (DNN) inference has emerged as a promising approach for delivering intelligent services to resource-constrained mobile devices.

Hugging Face Trending Papers
Aug 6

BALANCE: Hybrid Autoregressive-Speculative LLM Inference in Wireless Edge Networks

Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks. LLM inference mainly relies on two approaches: Autoregressive decoding (AD) generates output tokens sequentially, resulting in long latency; Speculative decoding (SD) accelerates inference by using a small language model (SLM) to generate multiple draft tokens for LLM verification, but incurs extra memory costs.

arXiv AI
Jul 21

SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs

arXiv:2607. 18081v1 Announce Type: cross Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across a range of Natural Language Processing (NLP) tasks, but their high computational and memory demands pose significant challenges for deployment on resource-constrained edge devices.

By Huzaifa Shaaban Kabakibo, Eric Schniedermeyer, Artem Burchanow, Lin Wang
arXiv Machine Learning
Jul 23

Self-organizing Architecture of Receptron Units: a Hardware-Aware Framework for Edge Intelligence

arXiv:2607. 20162v1 Announce Type: new Abstract: The growing demand for intelligent processing at the edge of IoT networks is constrained by the severe computational and memory limitations of microcontroller units, which render impractical conventional deep learning approaches.

By Stefano Radice, Ludovico Casaccia, Riccaro Emanuele Beccalli, Bruno Paroli, Paolo Milani