arXiv Machine Learning July 28, 2026 By Shuo Wang, Fang Xi, Wenyuan Huang, Qing Wang, Junming Su

KAP: Bridging the Knowledge Selection-Runtime Consumption Gap in LLM Systems

Read the original on arXiv Machine Learning →

arXiv:2607. 24260v1 Announce Type: new Abstract: Modern LLM systems increasingly rely on knowledge-selection processes that produce high-value structured priors, such as ranked evidence, graph topology, multimodal alignment, and confidence signals.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 16

A Spatio-Temporal Expert Prefetching Framework for Efficient MoE-based LLM Inference

arXiv:2606. 15453v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) based large language models (LLMs), such as Qwen and DeepSeek, have recently emerged as an effective approach to improving model capacity without proportionally increasing computational cost.

By Yingnan Zhao, Razvan Bunescu, Ahmed Louri, Avinash Karanth, Ke Wang

llms

arXiv AI

Aug 14

Dual-Flow Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation

arXiv:2608. 12385v1 Announce Type: new Abstract: As large language models serve more requests, cumulative inference cost is becoming increasingly important relative to one-time training cost.

By Liming Liu, Mingze Wang, Tuo Zhao

llms rag efficiency

arXiv AI

Sep 30

Adaptive Mass-Segmented KV Compression for Long-Context Reasoning

arXiv:2605.23200v2 Announce Type: replace-cross Abstract: The linear growth of the Key-Value (KV) cache is a critical bottleneck in long-form LLM inference. Existing KV compression methods mitigate t...

By Junzhe Yang, Xiaoyu Shen

llms efficiency

arXiv AI

Jul 14

Turbo Connection: Reasoning as Information Flow from Higher to Lower Layers

arXiv:2602. 17993v2 Announce Type: replace-cross Abstract: Complex problems, whether in math, logic, or planning, are solved by humans through a sequence of steps where the result of one step informs the next.

By Mohan Tang, Sidi Lu

llms fine-tuning benchmarks

arXiv Machine Learning

Sep 17

GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inference

GroupKV is a lightweight hierarchical KV cache management system designed for long‑context diffusion large language model (dLLM) inference. It partitions the context into contiguous groups and uses coarse‑to‑fine sparse selection, cross‑layer consistency for predictive prefetching, and a staleness correction mechanism to keep the cache coherent amid dynamic KV updates. The approach also incorporates streaming prefill to lower peak memory usage, achieving up to 48× longer serviceable context, 3.73× faster inference in offload‑based settings, and competitive task accuracy.

By Jinhao Wang, Zhexin Hu, Kangjie Zhou, Xin Zhou, Fangfang Liu

llms diffusion efficiency

arXiv AI

Aug 25

SAEM: Stage-Aware Expert Management for Memory-Efficient MoE Inference in Chain-of-Thought Reasoning

arXiv:2608.21614v1 Announce Type: new Abstract: Chain-of-thought (CoT) prompting improves LLM reasoning by decomposing complex problems into intermediate steps, but its sequential nature increases de...

By Yujie Zhang, Bin Gao, Tulika Mitra

llms efficiency benchmarks

One email a morning, machine-written

Related stories

A Spatio-Temporal Expert Prefetching Framework for Efficient MoE-based LLM Inference

Dual-Flow Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation

Adaptive Mass-Segmented KV Compression for Long-Context Reasoning

Turbo Connection: Reasoning as Information Flow from Higher to Lower Layers

GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inference

SAEM: Stage-Aware Expert Management for Memory-Efficient MoE Inference in Chain-of-Thought Reasoning