arXiv AI

Multi-primitive in-memory computing for Monte Carlo tree search

arXiv:2607. 22869v1 Announce Type: cross Abstract: Monte Carlo tree search (MCTS) enables artificial intelligence (AI) decision-making, but requires 55-300 W on conventional processors, limiting edge deployment.

arXiv AI

Aug 10

Predictive Multi-Tier Memory Management for KV Cache in Large-Scale GPU Inference

arXiv:2604. 26968v2 Announce Type: replace-cross Abstract: Key-value (KV) cache memory management is the primary bottleneck limiting throughput and cost-efficiency in large-scale GPU inference serving.

By Sanjeev Rao Ganjihal

agents efficiency

arXiv AI

Jun 3

CRAM-ER: Error-Resilient Spintronic Computational Random Access Memory for Scalable In-Memory Computation

arXiv:2606. 02781v1 Announce Type: cross Abstract: Deep neural networks (DNNs) have achieved state-of-the-art performance across diverse domains.

By Sohan Salahuddin Mugdho, Md. Shahedul Hasan, Brahmdutta Dixit, Yang Lv, Jian-Ping Wang, Cheng Wang

fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 16

Optimizing Binary and Ternary Neural Network Inference on RRAM Crossbars using CIM-Explorer

arXiv:2505. 14303v3 Announce Type: replace-cross Abstract: Using Resistive Random Access Memory (RRAM) crossbars in Computing-in-Memory (CIM) architectures offers a promising solution to overcome the von Neumann bottleneck.

By Rebecca Pelke, Jos\'e Cubero-Cascante, Nils Bosbach, Niklas Degener, Florian Idrizi, Lennart M. Reimann, Jan Moritz Joseph, Rainer Leupers

efficiency

arXiv Machine Learning

Jul 30

LLMET: Enabling Cross-Layer Evaluation of Emerging M3D Memories for Energy-Efficient LLM Serving

arXiv:2607. 26491v1 Announce Type: cross Abstract: The energy consumption of Large Language Model (LLM) serving is becoming a major system challenge as deployment scales, driven by hardware power and thermal constraints and rising electricity costs.

By Ming-Yen Lee, Hanchen Yang, Faaiq Waqar, Harsono Simka, Tushar Krishna, Muhammed Ahosan Ul Karim, Shimeng Yu

llms

arXiv AI

Jul 7

Optimizing ML Workload Partitioning between CPUs and CIM Accelerators for Heterogeneous Computing

arXiv:2607. 05240v1 Announce Type: cross Abstract: Computing-in-Memory (CIM) accelerators execute Matrix-Vector Multiplications (MVMs) in memory, making them a compelling solution for Machine Learning (ML) workloads.

By Joel Klein, Rebecca Pelke, Roberto Laudani, Jan Moritz Joseph, Rainer Leupers

efficiency

arXiv AI

Jul 1

An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU

arXiv:2603. 16428v2 Announce Type: replace-cross Abstract: Fine-tuning Large Language Models (LLMs) has become essential for domain adaptation, but its memory-intensive property exceeds the capabilities of most GPUs.

By Ruijia Yang, Zeyi Wen

llms fine-tuning efficiency

arXiv Machine Learning

Jul 14

FastTPS: An Optimized Method for LLM Token Phase for AI accelerators

arXiv:2607. 11211v1 Announce Type: new Abstract: The popularity of large language models (LLMs) escalates an ongoing demand for effective inference.

By Wenzong Yang, Danyang Zhang, Kun Cao, Tejus Siddagangaiah, Rajeev Patwari, Zhanxing Pu, Siyin Kong, Zijiang Yang, Hao Zhu, Varun Sharma, Yue Gao, Tianping Li, Fan Yang, Jicheng Chen, Yushan Chen, Fennian Zhao, Aaron Ng, Elliott Delaye, Ashish Sirasao, Sudip Nag

llms efficiency

arXiv Machine Learning

Jun 19

Low-Energy Reduced RISC-V Instruction Subset Processor for Tsetlin Machine Inference at the Edge

arXiv:2606. 19964v1 Announce Type: new Abstract: Tsetlin Machine (TM) is a logic-based machine learning approach that relies on simple bitwise operations and finite-state automata, which makes it attractive for edge AI deployments.

By Chanda Gupta, Sanidhya Bhatia, Shaurya Priyadarshi, Himani Panwar, Rishad Shafik, Sudip Roy

arXiv Machine Learning

Jul 27

HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding

arXiv:2607. 22389v1 Announce Type: cross Abstract: With the rapid adoption of long-context large language models (LLMs), the continuously growing KV cache during decoding has become the critical memory bottleneck.

By Chao Fang, Jun Yin, Man Shi, Marian Verhelst

llms efficiency benchmarks

arXiv Machine Learning

Jun 3

Multi-Segment Attention: Enabling Efficient KV-Cache Management for Faster Large Language Model Serving

arXiv:2606. 02964v1 Announce Type: cross Abstract: Large Language Model (LLM) inference relies on key-value (KV) caches to avoid redundant attention computation.

By Chunan Shi, Yilei Chen, Yilin Chen, Xupeng Miao, Bin Cui

llms rag agents efficiency

arXiv AI

Jul 13

STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU

arXiv:2607. 09385v1 Announce Type: cross Abstract: The growing adoption of large language model-based agents within operating system workflows has increased the importance of energy-efficient inference on laptop-class systems-on-chip (SoCs).

By Victor J. B. Jung, Gagandeep Singh, Joseph Melber, Kristof Denolf, Francesco Conti, Luca Benini

llms agents efficiency safety

arXiv Machine Learning

Jun 8

OpenACMv2: An Accuracy-Constrained Co-Optimization Framework for Approximate DCiM

arXiv:2603. 13042v2 Announce Type: replace Abstract: Digital Compute-in-Memory (DCiM) accelerates neural networks by reducing data movement.

By Yiqi Zhou, Yue Yuan, Yikai Wang, Bohao Liu, Qinxin Mei, Zhuohua Liu, Shan Shen, Wei Xing, Daying Sun, Li Li, Guozhu Liu

Related stories