Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

3,245 stories · RSS feed

arXiv Machine Learning

Jul 28

Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation

arXiv:2607. 23125v1 Announce Type: new Abstract: Post-training enables vision-language models (VLMs) to understand human instructions and perform various downstream tasks.

By Shuai Wang, Daoan Zhang, Zhe Tang, Hao Cheng, Jiaheng Wei

llms reinforcement-learning efficiency multimodal benchmarks

arXiv Machine Learning

Jul 28

Learning Reusable Hybrid Motion Priors for Humanoid Locomotion from Motion Imitation

arXiv:2607. 24083v1 Announce Type: new Abstract: Reinforcement learning can produce robust humanoid controllers, but each new task is typically trained as a separate policy with its own reward design and training process.

By Valerio Belli (UNIROMA, UCL), Valerio Modugno (UCL), Enrico Mingo Hoffman (HUCEBOT), Fabio Amadio (HUCEBOT)

reinforcement-learning robotics efficiency

arXiv AI

Jul 28

Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment

arXiv:2508. 02929v3 Announce Type: replace-cross Abstract: Scaling laws have been established for recommender systems, yet efficiently deploying foundation model (FM) across multiple recommendation surfaces remains a major unsolved challenge.

By Dai Li, Kevin Course, Wei Li, Hongwei Li, Jie Hua, Yiqi Chen, Zhao Zhu, Rui Jian, Xuan Cao, Bi Xue, Yu Shi, Jing Qian, Kai Ren, Matt Ma, Qunshu Zhang, Rui Li

rag fine-tuning efficiency

arXiv AI

Jul 28

Hierarchical Group-Conditional Conformal Risk Control for Selective Prediction in Language Models

arXiv:2607. 24562v1 Announce Type: new Abstract: Large language models serve heterogeneous populations structured by domain, topic difficulty, and linguistic style.

By Murilo Salem, Lu\'isa B\"ohm, Daniel Pontes, Anderson Ferrugem

llms efficiency benchmarks

arXiv Machine Learning

Jul 28

Outcome-Confounded Local Supervision in On-Policy Distillation

arXiv:2607. 23731v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student on its own trajectories while a teacher supplies dense token-level likelihoods at student-visited prefixes.

By Guoqing Ma

efficiency safety

arXiv AI

Jul 28

A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inference

arXiv:2607. 24148v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have demonstrated strong potential for embodied AI, yet their high inference latency on GPUs limits real-time deployment.

By Zhuoran Song, Haozhe Jiang, Chunyu Qi, Minnan Pei, Gang Li, Xiaoyao Liang, Haibing Guan

robotics efficiency multimodal

arXiv AI

Jul 28

scMIR: a vision-language foundation model for single-cell light microscopy image representation

arXiv:2607. 22712v1 Announce Type: cross Abstract: Single-cell light microscopy images have become an important data source for characterizing cell phenotypes, but their complexity and heterogeneity pose challenges to high-throughput automated analysis.

By Yifan Shang (Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong, China, College of Computer Science and Electronic Engineering, Hunan University, Changsha, China), Jiahui Tan (College of Computer Science and Electronic Engineering, Hunan University, Changsha, China), Xiangxiang Zeng (College of Computer Science and Electronic Engineering, Hunan University, Changsha, China), Renjie Zhou (Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong, China)

fine-tuning efficiency multimodal benchmarks safety

arXiv Machine Learning

Jul 28

CausalGate: Causal Importance Distillation for Transformer Module Pruning

arXiv:2607. 22720v1 Announce Type: new Abstract: Existing adaptive inference methods for Large Language Models rely on observational heuristics, such as hidden-state similarity or activation magnitudes, to drop redundant modules.

By Kiran Nair, Smriti Regmi, Rodrigue Rizk

llms efficiency benchmarks

arXiv AI

Jul 28

PRESTO: Prefix-Aligned Tree Drafting for Diffusion Speculative Decoding

arXiv:2607. 22634v1 Announce Type: new Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs, generating tokens in parallel.

By Zheng Wang, Zhifan Ye, Qi Cheng, Yonggan Fu, Ziyan Wang, Feng Zhu, Haozhe Zhao, Jan Kautz, Pavlo Molchanov, Humphrey Shi, Minjia Zhang

llms diffusion efficiency benchmarks

arXiv Machine Learning

Jul 28

Analyzing the Importance of Blank for CTC-Based Knowledge Distillation

arXiv:2506. 01503v2 Announce Type: replace Abstract: With the rise of large pre-trained foundation models for automatic speech recognition new challenges appear.

By Benedikt Hilmes, Nick Rossenbach, Ralf Schl\"uter

efficiency multimodal

arXiv AI

Jul 28

CachedSearch: Training-Free Cached Exploration for Test-Time Search in Video Diffusion

arXiv:2607. 23159v1 Announce Type: new Abstract: Test-time search lets small video diffusion models rival larger ones, but costs 2-10x more.

By Shreshth Saini, Neil Birkbeck, Yilin Wang, Balu Adsumilli, Alan C. Bovik

diffusion efficiency

arXiv Machine Learning

Jul 28

Gleam: Adaptive Network-Efficient CUDA API Remoting for Cross-Device GPU Sharing over LANs

arXiv:2607. 23115v1 Announce Type: cross Abstract: This paper aims to enable computation- and communication-efficient GPU sharing across devices within local area networks (LANs), facilitating ubiquitous AI inference on heterogeneous personal devices.

By Zhihao Xu, Hao Zhong, Zeting Zhou, Yuhang Xu, Haoyu Tong, Wei Wang, Jinshan Chen, Keqiang He, Chong Zhu, Shengzhong Liu, Fan Wu, Guihai Chen

efficiency benchmarks

arXiv AI

Jul 28

From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search

arXiv:2607. 24280v1 Announce Type: new Abstract: Agentic search enables large language models to solve knowledge-intensive tasks by interleaving multi-step reasoning with retrieval, yet optimizing this with outcome-based reinforcement learning (RL) provides only sparse supervision.

By Junlin Liu, Jiangwang Chen, Zixin Song, Shuaiyu Zhou, Chunji Lv, Hank Wu, Kailin Jiang, Jinyang Wu, Bohan Yu, Chenxi Zhou

llms agents reinforcement-learning efficiency benchmarks

arXiv AI

Jul 28

Masked Distillation: Internalizing the Chain-of-Thought in Language Models

arXiv:2607. 22629v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) produce long, explicit chains of intermediate steps before generating a final answer at inference time.

By Durgesh Kalwar, Vardhan Palod, Subbarao Kambhampati

llms efficiency

arXiv Machine Learning

Jul 28

FusionML: Prefill, Not Decode - Mechanism and Boundaries of CPU+GPU Co-Execution on Unified-Memory Apple Silicon

arXiv:2607. 22785v1 Announce Type: cross Abstract: Apple-Silicon SoCs share CPU, GPU, and Neural Engine over one unified memory system, raising the question of whether transformer inference can be accelerated by splitting single operators across units.

By Om Mohite

llms efficiency

arXiv AI

Jul 28

PathScale-R1: Cross-scale Reasoning for Pathological Image Analysis

arXiv:2607. 23794v1 Announce Type: cross Abstract: Pathological diagnosis is inherently multi-scale, requiring the integration of global tissue architecture at low magnification with cellular morphology at higher magnification.

By Chi Phan, Tianyi Zhang, Yufeng Wu, Qiaochu Xue, Jiajie Zhang, Linghan Cai, Zeyu Liu, Sudong Wang, Yueming Jin, Dan Hu

llms nlp reinforcement-learning fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Jul 28

SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agents

arXiv:2607. 23263v1 Announce Type: new Abstract: Deciding whether a trajectory actually fulfills its instruction governs how we measure computer-use agents on long-horizon graphical-user-interface tasks and how we train them with reinforcement learning.

By Yang Wan, Zhenhao Zhang, Jierui Wang, Linchao Zhu

agents reinforcement-learning efficiency

arXiv AI

Jul 28

CuraWeb: Joint Optimization of Quality, Redundancy, and Diversity for Web-Scale Pretraining Data

arXiv:2607. 22662v1 Announce Type: new Abstract: Open-web corpora curated via highly selective filters, such as FineWeb-Edu and DCLM, constitute the core of LLM pretraining data and have significantly advanced LLM performance.

By Peiguang Li, Yongwei Zhou, Juncheng Diao, Yuchun Fan, Jian Yang, Jianxiao Yang, Zhongda Su, Shuguang Jiao, Xiao Wei, Zhiye Zou, Gan Dong, Zhizhao Zeng, Rongxiang Weng, Jingang Wang, Xunliang Cai

llms efficiency benchmarks

arXiv Machine Learning

Jul 28

When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation

arXiv:2607. 23390v1 Announce Type: new Abstract: When can additional low-bit residual computation replace missing numerical precision for a fixed input-output map?

By Mojtaba Soltanalian

efficiency

arXiv Machine Learning

Jul 28

LLM-based Source Code Compression via Thresholded Symbol Ranking

arXiv:2607. 24192v1 Announce Type: cross Abstract: We study the problem of lossless compression of source code, motivated by the storage demands of large-scale software archives, such as Software Heritage (https://www.

By Angelo Nardone, Paolo Ferragina

llms efficiency

Get the daily briefing