Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

3,411 stories · RSS feed

arXiv AI

Jun 30

Internalized Reasoning for Long-Context Visual Document Understanding

arXiv:2604. 02371v2 Announce Type: replace-cross Abstract: Visual long-document understanding is critical for enterprise, legal, and scientific applications, yet the best performing open recipes have not explored reasoning, a capability which has driven leaps in math and code performance.

By Austin Veselka

llms fine-tuning efficiency data-engineering

arXiv AI

Jun 30

Monte Carlo Query Search: Active Capability Assessment of AI Agents

arXiv:2512. 16733v3 Announce Type: replace Abstract: Black-box AI (BBAI) systems, including foundation-model agents, are increasingly used for sequential decision making.

By Daniel Bramblett, Rushang Karia, Adrian Ciotinga, Pulkit Verma, YooJung Choi, Siddharth Srivastava

agents efficiency

arXiv Machine Learning

Jun 30

TimeLAVA: Learning-Agnostic Valuation for Time Series Data

arXiv:2606. 18729v3 Announce Type: replace-cross Abstract: Data valuation quantifies the intrinsic quality of individual samples to enable principled data curation, quality control, and robust learning.

By Wenqin Liu, Weizhi Quan, Aoqi Zuo, Erdun Gao, Vu Nguyen, Dino Sejdinovic, Howard Bondell, Mingming Gong

efficiency

arXiv Machine Learning

Jun 30

HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers

arXiv:2603. 12222v2 Announce Type: replace-cross Abstract: Vision Transformers require significant computational resources and memory bandwidth, severely limiting their deployment on resource-constraint hardware.

By Andy Li, Aiden Durrant, Milan Markovic, Georgios Leontidis

llms fine-tuning efficiency

arXiv Machine Learning

Jun 30

SparsePixels: Efficient Convolution for Sparse Data on FPGAs

arXiv:2512. 06208v3 Announce Type: replace-cross Abstract: Inference of standard convolutional neural networks (CNNs) on FPGAs often incurs high latency and a long initiation interval due to the deep nested loops required to densely convolve every input pixel regardless of its feature value.

By Ho Fung Tsoi, Dylan Rankin, Vladimir Loncar, Philip Harris

efficiency

arXiv Machine Learning

Jun 30

Attention Enhanced Entity Recommendation for Intelligent Monitoring in Cloud Systems

arXiv:2510. 20640v2 Announce Type: replace Abstract: In this paper, we present DiRecGNN, an attention-enhanced entity recommendation framework for monitoring cloud services at Microsoft.

By Fiza Husain, Anson Bastos, Anjaly Parayil, Ayush Choure, Chetan Bansal, Rujia Wang, Saravan Rajmohan

llms efficiency benchmarks

arXiv Machine Learning

Jun 30

One-Step Gradient Delay is Not a Barrier for Large-Scale Asynchronous Pipeline Parallel LLM Pretraining

arXiv:2606. 30634v1 Announce Type: new Abstract: Modern large-scale LLM pretraining benefits from utilizing Pipeline Parallelism; however, synchronous implementations leave GPUs idle during pipeline bubbles, wasting computational resources.

By Philip Zmushko, Egor Petrov, Nursultan Abdullaev, Mikhail Khrushchev, Samuel Horv\'ath

llms efficiency

arXiv Machine Learning

Jun 30

Few-Step Boltzmann Generators via Scalable Likelihood Flow Maps

arXiv:2606. 29110v1 Announce Type: new Abstract: Recent progress in flow-based generative modeling has led to models that output high-quality samples while using only a small number of function evaluations.

By RuiKang OuYang, Hanlin Yu, Xinyue Ai, Yutong He, Nicholas M. Boffi, Pradeep Ravikumar, Jose Miguel Hernandez-Lobato, Max Simchowitz, Benjamin Kurt Miller, Omar Chehab

efficiency benchmarks

arXiv AI

Jun 30

UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation

arXiv:2603. 22282v2 Announce Type: replace-cross Abstract: We present UniMotion, to our knowledge the first unified framework for simultaneous understanding and generation of human motion, natural language, and RGB images within a single architecture.

By Ziyi Wang, Xinshun Wang, Shuang Chen, Yang Cong, Mengyuan Liu

llms nlp efficiency multimodal benchmarks safety

arXiv AI

Jun 30

Agile Reinforcement Learning through Separable Neural Architecture and Applications

arXiv:2601. 23225v2 Announce Type: replace-cross Abstract: Deep reinforcement learning (RL) is increasingly deployed in resource-constrained environments, yet go-to function approximators - multilayer perceptrons (MLPs) - are often parameter-inefficient due to an imperfect inductive bias for the smooth structure of many value functions.

By Rajib Mostakim, Reza T. Batley, Sourav Saha

reinforcement-learning efficiency benchmarks safety

arXiv AI

Jun 30

Instance-Conditioned Adaptation for Large-scale Generalization of Neural Routing Solver

arXiv:2405. 01906v3 Announce Type: replace Abstract: In modern intelligent transportation systems (ITS), particularly in freight transportation and logistics, real-time route planning is crucial.

By Changliang Zhou, Xi Lin, Zhenkun Wang, Xialiang Tong, Mingxuan Yuan, Qingfu Zhang

efficiency benchmarks

arXiv AI

Jun 30

Automating the Design of Embodied AgentArchitectures

arXiv:2606. 30111v1 Announce Type: cross Abstract: Embodied agents are typically built as hand-designed compositions of perception, memory, planning, and action modules.

By Jian Zhou, Sihao Lin, Jin Li, Shuai Fu, Gengze Zhou, Qi Wu

agents nlp robotics efficiency multimodal

arXiv AI

Jun 30

Neural Subspace Reallocation: Continual Learning as Retrieval-Based Subspace Memory Management

arXiv:2606. 30067v1 Announce Type: cross Abstract: We introduce Neural Subspace Reallocation (NSR), which reframes continual learning as memory management over parameter subspaces.

By Byeong Hoon Yoon

rag fine-tuning efficiency benchmarks

arXiv AI

Jun 30

Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM

arXiv:2606. 29563v1 Announce Type: cross Abstract: Large language models (LLMs) excel at complex tasks like question answering and summarization, thanks to their ability to handle long-context inputs.

By Shuvendu Roy, Mengyao Zhai, Hossein Hajimirsadeghi, Golnoosh Samei

llms nlp efficiency

arXiv AI

Jun 30

Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation

arXiv:2606. 29464v1 Announce Type: cross Abstract: Vision-language dataset distillation (VLDD) compresses a large image-text paired dataset into a small set of synthetic pairs that can efficiently train contrastive vision-language models under strict data and compute budgets.

By Jongoh Jeong, Sun-Kyung Lee, Kuk-Jin Yoon

llms rag efficiency multimodal benchmarks safety

arXiv AI

Jun 30

Learning to Adaptively Allocate Gaussians for Arbitrary-Scale Image Super-Resolution

arXiv:2606. 29400v1 Announce Type: cross Abstract: In computer graphics, visual content is continuously warped, zoomed and resampled.

By Giulio Federico, Giuseppe Amato, Claudio Gennaro, Fabio Carrara, Marco Di Benedetto

diffusion efficiency multimodal benchmarks

arXiv AI

Jun 30

Fast Enough to Act: Spatio-Temporal Visual Token Merging for Low-Latency Robotic VLMs and VLAs

arXiv:2606. 29350v1 Announce Type: cross Abstract: Vision-language models and vision-language action models endow the robot with unprecedented capabilities.

By Junzhou Chen, Jindong Wang, Gang Zhou

llms nlp robotics efficiency multimodal

arXiv AI

Jun 30

Deterministic Decisions for High-Stakes AI. A Zero-Egress Pipeline with the Deployability of RAG and the Accuracy of Machine Learning

arXiv:2606. 29280v1 Announce Type: cross Abstract: We identify intervention bias as a previously unquantified failure mode of zero-shot large-language-model (LLM) educational advisory agents: without task-specific training, they recommend action when a hindsight-optimal oracle policy mandates inaction.

By Craig Atkinson

llms rag agents efficiency benchmarks safety data-engineering

arXiv AI

Jun 30

KernelSight-LM: A Kernel-Level LLM Inference Simulator

arXiv:2606. 28565v1 Announce Type: cross Abstract: As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to meet cost and latency targets.

By Xiteng Yao, Taeho Kim, Hengzhi Pei, Xinle Liu, Kyle Ulrich, Leonard Lausen, Ashish Khetan, Xiang Song, George Karypis, Martin Herbordt

llms efficiency benchmarks

arXiv Machine Learning

Jun 30

HyperDFlash: Hyper-Connection-Aligned Block Speculative Decoding with Gated Residual Reduction

arXiv:2606. 26744v2 Announce Type: replace Abstract: We present HyperDFlash, a block-parallel speculative decoding framework tailored to DeepSeek-V4's Hyper-Connections (HC).

By Luxi Lin, Shuang Peng, Rui Ma, Junhao Hua, Shuwei Fan, Zhengda Qin, Qiang Wang, Hongjian Sun, Fangmin Chen, Songwei Liu

efficiency benchmarks safety

Get the daily briefing