Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

6,436 stories · RSS feed

arXiv AI

Aug 12

Fast and Memory-Efficient Wavelet Convolutions via I/O-Aware Reformulation

arXiv:2608. 10805v1 Announce Type: cross Abstract: Wavelet convolution (WTConv) has emerged as an increasingly popular drop-in replacement for standard convolutions, expanding a network's receptive field exponentially with the number of decomposition levels while keeping the parameter count linear.

By Amit Aflalo, Shahaf E. Finder, Roy Amoyal, Eran Treister, Oren Freifeld

efficiency

arXiv AI

Aug 12

Rethinking Text-Based Image Retrieval in Specific Domain

arXiv:2608. 10524v1 Announce Type: cross Abstract: Driven by the rapid advancement of vision-language representation learning, Text-based Image Retrieval (TBIR) has made notable progress.

By Jingyang Tan, Sheng Yang, Yuanpeng Chen, Jian Wang, Nianjin Ye, Chen Xing, Lanpeng Jia

fine-tuning efficiency multimodal benchmarks

arXiv Machine Learning

Aug 12

MoE Proxy Models for Low-Cost Failure Reproduction and Diagnosis in LLM RL Post-Training

arXiv:2608. 10823v1 Announce Type: new Abstract: Reinforcement learning (RL) post-training of large language models (LLMs) is computationally intensive and involves complex system pipelines with substantial debugging overhead.

By Yikai Wang, Chuansai Zhou, Yuhang Zhou, Weiqiang Wu, Cong Wu, Yue Deng, Ben Feng, Mingming Zhu, Beirong Zhou, Zhibin Wang, Sheng Zhong, Chen Tian, Wangze Zhang

llms reinforcement-learning efficiency

arXiv AI

Aug 12

LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning

arXiv:2601. 10129v2 Announce Type: replace-cross Abstract: Current multimodal latent reasoning often relies on external supervision (e.

By Linquan Wu, Tianxiang Jiang, Yifei Dong, Haoyu Yang, Fengji Zhang, Shichaang Meng, Ai Xuan, Linqi Song, Jacky Keung

llms rag efficiency multimodal

arXiv AI

Aug 12

Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critique

arXiv:2608. 10430v1 Announce Type: cross Abstract: Large Language Models (LLMs) deployed as AI agents frequently exhibit user specification-grounding failures, executing hallucinated, undesired actions to force a resolution rather than expressing uncertainty.

By Sanidhya Vijayvargiya, Rahul Lokesh

llms agents diffusion robotics fine-tuning efficiency safety

arXiv Machine Learning

Aug 12

ReOrder-OPD:Reliability-Aware Prompt Ordering for On-Policy Distillation

arXiv:2608. 10905v1 Announce Type: new Abstract: On-policy distillation (OPD) applies token-level teacher supervision to student-generated trajectories, but this supervision is not always reliable.

By Ximo Zhu, Ruiqi Liu, Rong Wang, Ping Wu, Xiang Zheng, Wenzhuo Xu, Xubin Yao, Zhiyuan Yan, Bo Li, Jun Gao, Xiaolei Lv

efficiency

arXiv Machine Learning

Aug 12

Scheduling Mixed RL Rollouts Beyond Prefix Locality

arXiv:2608. 11152v1 Announce Type: cross Abstract: Modern reinforcement learning (RL) post-training pipelines for large language models (LLMs) increasingly combine rollout workloads across multiple domains and feedback paradigms.

By Zetao Hong, Song Yuan, Yuanhao Ding, Yibo Zhu, Daxin Jiang, Zhibin Wang, Chen Tian

llms agents reinforcement-learning efficiency

arXiv AI

Aug 12

Decodable But Not Detachable: Training Data Granularity Determines Parametric Modularity in Large Language Models

arXiv:2608. 10214v1 Announce Type: new Abstract: Do large language models contain domain-specific parametric shells: concentrated, causally necessary neuron populations whose removal selectively degrades a target domain while sparing others?

By Marcus Armstrong, Navid Ayoobi, Arjun Mukherjee

llms efficiency

arXiv Machine Learning

Aug 12

Conditional Independence Tests for Constraint-Based Causal Discovery: A Survey

arXiv:2608. 11156v1 Announce Type: cross Abstract: Conditional Independence (CI) tests are the statistical engine of constraint-based causal discovery: in algorithms such as PC (Peter-Clark) and FCI (Fast Causal Inference), skeleton pruning and key orientations follow directly from CI decisions.

By Pavel Averin, Theodoros Moysiadis, Ioannis Katakis

efficiency

arXiv AI

Aug 12

Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation

arXiv:2608. 11191v1 Announce Type: cross Abstract: GUI Visual Grounding is a fundamental capability for GUI agents.

By Shiyu Xuan, Zechao Li

agents reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Aug 12

Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation

arXiv:2608. 10392v1 Announce Type: new Abstract: Mixture-of-experts (MoE) models have recently moved beyond routing a fixed number of complete experts.

By Gongli Zhang, Zhulin Liu, C. L. Philip Chen

rag efficiency

arXiv Machine Learning

Aug 12

CurveFP: Rational-Radix Logarithmic Datatypes with Closed Products for Language Models

arXiv:2608. 10010v1 Announce Type: new Abstract: Low-precision datatypes reduce language-model cost, but most formats optimize scalar fidelity while leaving the arithmetic induced by their products unchanged.

By Ye Qiao

llms efficiency

arXiv AI

Aug 12

Lost in Reconstruction: Aligning Action Representations with Language in Vision-Language-Action Models

arXiv:2608. 10484v1 Announce Type: cross Abstract: Action verbs describe not only the physical outcomes of actions, but also how those actions are performed.

By Li Wenjie, Yash Jangir, Ignacy Stepka, Yash Agarwal, Marion Kipsang, Yonatan Bisk

llms nlp robotics efficiency multimodal

arXiv AI

Aug 12

Bandwidth-Efficient Multi-Agent Communication through Information Bottleneck and Vector Quantization

arXiv:2602. 02035v2 Announce Type: replace-cross Abstract: Multi-agent reinforcement learning systems deployed in real-world robotics applications face severe communication constraints that significantly impact coordination effectiveness.

By Ahmad Farooq, Kamran Iqbal

agents reinforcement-learning robotics efficiency

arXiv AI

Aug 12

Putting Registers to Work: Task Registers for Token Pruning in Vision Transformers

arXiv:2608. 10989v1 Announce Type: cross Abstract: Token-pruning policies are usually designed for a single recognition pipeline, but pretrained Vision Transformers are reused across tasks with different spatial demands.

By Hongsen Cao, Mona Jaber, Shanxin Yuan, Ahmed Sayed

llms computer-vision efficiency

arXiv AI

Aug 12

GeoForge: Non-Parametric Self-Evolving Agents for Earth-Observation Reasoning

arXiv:2608. 10494v1 Announce Type: new Abstract: Earth observation (EO) agents construct scientifically valid tool workflows and ground their conclusions in current geospatial evidence.

By Xin Xiao, Jiang Zhong, Junnan Zhu, Yingchao Feng, Peijin Wang, Yidan Zhang, Kaiwen Wei

llms agents efficiency benchmarks safety

arXiv Machine Learning

Aug 12

SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Features

arXiv:2608. 10709v1 Announce Type: new Abstract: Quantization-Aware Training (QAT) enables the deployment of quantized models with minimal accuracy degradation.

By HyeonJun Lee, Hyeonsik Jo, Jinwoo Chung, Jangho Kim

efficiency safety

arXiv Machine Learning

Aug 12

Reinforcement Learning-based Semi-supervised Knowledge Distillation with LLM-as-a-Judge

arXiv:2604. 02621v2 Announce Type: replace-cross Abstract: Reinforcement Learning (RL) substantially improves the reasoning capabilities of language models, but most existing RL fine-tuning approaches rely entirely on ground-truth verifiable rewards and thus labeled datasets with verifiable answers.

By Yiyang Shen, Lifu Tu, Weiran Wang

llms reinforcement-learning fine-tuning efficiency

arXiv AI

Aug 12

Hybrid Token Compression for Vision-Language Models

arXiv:2512. 08240v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) rely on hundreds of visual tokens, leading to high computational and memory costs.

By Jusheng Zhang, Xiaoyang Guo, Tongyu Mo, Qinhan Lv, Wenhao Chai, Jian Wang, Keze Wang, Liang Lin

llms efficiency multimodal benchmarks

arXiv Machine Learning

Aug 12

When Interpretability Is Unequally Distributed: Fairness in Hybrid Interpretable Models

arXiv:2605. 28626v2 Announce Type: replace Abstract: Hybrid interpretable models combine a transparent component with a black-box model by assigning some examples to the former and deferring the rest to the latter.

By Ziba Jabbar Zare, Ulrich A\"ivodji, Julien Ferry, Thibaut Vidal

efficiency benchmarks safety

Get the daily briefing