arXiv Machine Learning June 11, 2026 By Zhiyuan Cheng, Longying Lai

Energy-Efficient On-Device RAG on a Mobile NPU: System Design and Benchmark on Snapdragon X Elite

Read the original on arXiv Machine Learning →

arXiv:2606. 11257v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) pipelines are compute-intensive, combining embedding, retrieval, reranking, and large language model (LLM) generation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 8

Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference

arXiv:2607. 05475v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) on mobile devices enhances privacy and reduces latency, but is severely bottlenecked by hardware inefficiency.

By Guanyu Cai, Ruiming Tian, Lang Yang, Zhouhong Ren, Jinliang Yuan, Lingkun Li, Jiliang Wang

llms efficiency safety

arXiv AI

Sep 10

A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware

arXiv:2609.08307v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires bala...

By Maysam Khatib, Moysis Symeonides, Demetris Trihinas, George Pallis, Marios D. Dikaiakos

llms efficiency benchmarks

arXiv AI

Sep 10

RAGMark: A Comprehensive Framework for Benchmarking Retrieval-Augmented Generation Systems

arXiv:2609.05760v1 Announce Type: cross Abstract: We present RAGMark, a modular benchmarking framework for advanced Retrieval-Augmented Generation (RAG) systems targeting small-scale multi-GPU enviro...

By Zlatan Feric, Amir Taherin, Bin Ren, Yanzhi Wang, Jennifer Dy, David Kaeli

rag efficiency benchmarks

arXiv Machine Learning

Sep 10

EStream: Fast and Memory-Efficient MoE Prefill through Expert Virtualization on Mobile NPUs

arXiv:2609.06551v1 Announce Type: cross Abstract: Mobile vendors and application developers increasingly deploy LLMs on smartphones for diverse prefill-only services. Yet current systems rely mainly...

By Junming Zhang, Zhenzhe Zheng, Fan Wu, Xiaoyao Huang, Jie Wu

llms

arXiv AI

Jun 24

Does Mixture-of-Experts Actually Help Inference on Consumer and Edge Hardware? An Empirical Study

arXiv:2606. 21428v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) language models are often described as ideal for resource-constrained inference.

By Alfarizy Alfarizy, Hung Truong Thanh Nguyen, Ren\'e Richard, Roozbeh Razavi-Far, Hung Cao

llms efficiency benchmarks

arXiv AI

Jul 13

STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU

arXiv:2607. 09385v1 Announce Type: cross Abstract: The growing adoption of large language model-based agents within operating system workflows has increased the importance of energy-efficient inference on laptop-class systems-on-chip (SoCs).

By Victor J. B. Jung, Gagandeep Singh, Joseph Melber, Kristof Denolf, Francesco Conti, Luca Benini

llms agents efficiency safety

One email a morning, machine-written

Related stories

Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference

A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware

RAGMark: A Comprehensive Framework for Benchmarking Retrieval-Augmented Generation Systems

EStream: Fast and Memory-Efficient MoE Prefill through Expert Virtualization on Mobile NPUs

Does Mixture-of-Experts Actually Help Inference on Consumer and Edge Hardware? An Empirical Study

STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU