arXiv Machine Learning July 14, 2026 By Wenzong Yang, Danyang Zhang, Kun Cao, Tejus Siddagangaiah, Rajeev Patwari, Zhanxing Pu, Siyin Kong, Zijiang Yang, Hao Zhu, Varun Sharma, Yue Gao, Tianping Li, Fan Yang, Jicheng Chen, Yushan Chen, Fennian Zhao, Aaron Ng, Elliott Delaye, Ashish Sirasao, Sudip Nag

FastTPS: An Optimized Method for LLM Token Phase for AI accelerators

Read the original on arXiv Machine Learning →

arXiv:2607. 11211v1 Announce Type: new Abstract: The popularity of large language models (LLMs) escalates an ongoing demand for effective inference.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 17

Prefill/Decode-Aware Evaluation of LLM Inference on Emerging AI Accelerators

arXiv:2606. 17104v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly deployed in latency- and cost-sensitive settings, inference efficiency has become a central systems challenge.

By Shun Usami, Venkatram Vishwanath, E. Wes Bethel

llms efficiency

arXiv Machine Learning

Jul 27

HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding

arXiv:2607. 22389v1 Announce Type: cross Abstract: With the rapid adoption of long-context large language models (LLMs), the continuously growing KV cache during decoding has become the critical memory bottleneck.

By Chao Fang, Jun Yin, Man Shi, Marian Verhelst

llms efficiency benchmarks

arXiv AI

Jul 13

STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU

arXiv:2607. 09385v1 Announce Type: cross Abstract: The growing adoption of large language model-based agents within operating system workflows has increased the importance of energy-efficient inference on laptop-class systems-on-chip (SoCs).

By Victor J. B. Jung, Gagandeep Singh, Joseph Melber, Kristof Denolf, Francesco Conti, Luca Benini

llms agents efficiency safety

arXiv Machine Learning

Jul 31

WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning

arXiv:2607. 28418v1 Announce Type: cross Abstract: Pruning is a promising approach for improving the efficiency of LLMs.

By Haozhe Hu, Hao Wu, Peiran Yin, Chao Han, Yunpu Ma, Xiaoyu Shen

llms nlp efficiency benchmarks

arXiv Machine Learning

Jun 19

Low-Energy Reduced RISC-V Instruction Subset Processor for Tsetlin Machine Inference at the Edge

arXiv:2606. 19964v1 Announce Type: new Abstract: Tsetlin Machine (TM) is a logic-based machine learning approach that relies on simple bitwise operations and finite-state automata, which makes it attractive for edge AI deployments.

By Chanda Gupta, Sanidhya Bhatia, Shaurya Priyadarshi, Himani Panwar, Rishad Shafik, Sudip Roy

arXiv Machine Learning

Jun 3

Multi-Segment Attention: Enabling Efficient KV-Cache Management for Faster Large Language Model Serving

arXiv:2606. 02964v1 Announce Type: cross Abstract: Large Language Model (LLM) inference relies on key-value (KV) caches to avoid redundant attention computation.

By Chunan Shi, Yilei Chen, Yilin Chen, Xupeng Miao, Bin Cui

llms rag agents efficiency

Related stories

Prefill/Decode-Aware Evaluation of LLM Inference on Emerging AI Accelerators

HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding

STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU

WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning

Low-Energy Reduced RISC-V Instruction Subset Processor for Tsetlin Machine Inference at the Edge

Multi-Segment Attention: Enabling Efficient KV-Cache Management for Faster Large Language Model Serving