Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

3,411 stories · RSS feed

arXiv AI

Jun 30

Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL

arXiv:2604. 28123v3 Announce Type: replace-cross Abstract: The standard post-training recipe for large multimodal models (LMMs) applies supervised fine-tuning (SFT) on curated demonstrations followed by reinforcement learning with verifiable rewards (RLVR).

By Sudong Wang, Weiquan Huang, Xiaomin Yu, Zuhao Yang, Hehai Lin, Keming Wu, Chaojun Xiao, Chen Chen, Wenxuan Wang, Beier Zhu, Yunjian Zhang, Chengwei Qin

llms reinforcement-learning fine-tuning efficiency multimodal benchmarks safety

arXiv Machine Learning

Jun 30

When Does Online Imitation Learning Help in LLM Post-Training? The Role of (Non-)Realizability Beyond Horizon

arXiv:2606. 30445v1 Announce Type: new Abstract: Online imitation learning (IL), particularly on-policy distillation, has emerged as a strong LLM post-training approach, often outperforming offline supervised fine-tuning (SFT).

By Huaqing Zhang, Jingchu Gai, Juno Kim, Bingbin Liu, Andrej Risteski

llms fine-tuning efficiency

arXiv AI

Jun 30

DRIFT: Difficulty Routing Self-DIstillation with Rhythm-Gated Exploration and Success BuFfer Training

arXiv:2606. 30345v1 Announce Type: cross Abstract: Enabling large language models to achieve stable self-improvement without external expert supervision remains a central challenge in complex reasoning tasks.

By Haisen Luo, Yiwei Liu, Haoning Wang, Dan Liu, Junxi Yin, Haotian Wang, Lei Zhang, Xiaoyu Tian, Shuaiting Chen, Yuansheng Song, Baoyan Guo, Xiongfei Yan, Bolan Yang, Chengwei Liu, Ming Cui, Jiong Chen

llms reinforcement-learning efficiency benchmarks

arXiv AI

Jun 30

DuoMem: Towards Capable On-Device Memory Agents via Dual-Space Distillation

arXiv:2606. 29961v1 Announce Type: cross Abstract: Large Language Model (LLM)-based agents can solve complex procedural tasks by interacting with environments over multiple turns, but this ability typically depends on large models, long contexts, and repeated inference calls.

By Peyman Hosseini, Ondrej Bohdal, Ahmed Alajrami, Andrea Maracani, Ignacio Castro, Matthew Purver, Mete Ozay, Savas Ozkan, Taha Ceritli

llms agents robotics fine-tuning efficiency benchmarks

arXiv AI

Jun 30

Clustering Unsupervised Representations as Defense against Poisoning Attacks on Speech Commands Classification System

arXiv:2606. 28953v1 Announce Type: cross Abstract: Poisoning attacks entail attackers intentionally tampering with training data.

By Thomas Thebaud, Sonal Joshi, Henry Li, Martin Sustek, Jesus Villalba, Sanjeev Khudanpur, Najim Dehak

efficiency

arXiv AI

Jun 30

PHF: Privileged Hidden Flow for On-Policy Self-Distillation

arXiv:2606. 29340v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a reasoning model on rollouts sampled from its own policy by matching a privileged teacher that also sees verified reference solutions.

By Yuhan Li, Mingxu Zhang, Dazhong Shen, Ying Sun

efficiency

arXiv Machine Learning

Jun 30

Not All Objectives Are Born Equal: Priority-Constrained Descent for Hierarchical Multi-Objective Optimization

arXiv:2606. 29521v1 Announce Type: new Abstract: Deep learning problems rarely involve objectives that are equal in importance.

By Dara Varam, Mohamed I. Alhajri

efficiency

arXiv Machine Learning

Jun 30

Nonlinear mixture model motivated subspace clustering

arXiv:2606. 29261v1 Announce Type: new Abstract: We derive the linear union-of-subspaces (UoS) model for subspace clustering (SC) from the nonlinear mixture model (NMM) used in blind source separation (BSS) to represent a D-dimensional observation vector as an unknown multivariate nonlinear mapping of C latent variables.

By Ivica Kopriva

efficiency benchmarks

arXiv AI

Jun 30

Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning

arXiv:2508. 09883v2 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving.

By Xiaojun Wu, Xiaoguang Jiang, Huiyang Li, Jucai Zhai, Dengfeng Liu, Qiaobo Hao, Huang Liu, Zhiguo Yang, Ji Xie, Ninglun Gu, Jin Yang, Kailai Zhang, Yelun Bao, Jun Wang

llms reinforcement-learning fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 30

Beyond Trajectory Matching: Reflow with Marginal Distribution Alignment

arXiv:2606. 29287v1 Announce Type: new Abstract: Diffusion and continuous-flow generative models achieve high-quality generation, and their deterministic sampling can be formulated as solving learned ODE dynamics.

By Chen Wang, Peiran Yun, Pan Xie, Ke Deng

diffusion efficiency benchmarks safety

arXiv AI

Jun 30

JuZhou 1.0 Technical Report: The First Edge-Native Text-to-Image Foundation Model Trained Entirely on China-Developed AI Accelerators

arXiv:2606. 28421v1 Announce Type: cross Abstract: Text-to-image (T2I) diffusion models typically require substantial computational resources and cloud infrastructure, posing significant challenges for edge deployment in terms of latency, cost, and user privacy.

By Ce Chen, Congrui Wang, Yonglin Li, Zhenchen Wan, Mingyang Geng, Junhao Xiao, Zhengpeng Xing, Yaqing Hu, Yao Wu, Zhaoyang Qu, Long Lan, Xinwang Liu, Yingqi Peng, Shijia Li, Zufeng Zhang, Chen Ma, Jingjing Zhou, Xingyu Wang, Qilin Lu, Bin Jiang, Qilin Sun, Shanzhi Gu, Yaoguang Jin, Tongliang Liu, Kede Ma, Yifan Peng

llms diffusion efficiency multimodal safety

arXiv AI

Jun 30

GPC: Large-Scale Generative Pretraining for Transferable Motor Control

arXiv:2606. 29148v1 Announce Type: cross Abstract: Developing controllers capable of completing a wide range of tasks in a natural and life-like manner is a key challenge in enabling practical applications of physics-based character animation.

By Yi Shi, Yifeng Jiang, Chen Tessler, Xue Bin Peng

llms nlp reinforcement-learning fine-tuning efficiency

arXiv AI

Jun 30

DOPD: Dual On-policy Distillation

arXiv:2606. 30626v1 Announce Type: new Abstract: On-policy distillation (OPD) offers superior capacity transfer by supervising student-sampled trajectories with dense token-level signals.

By Xinlei Yu, Gen Li, Qingyi Si, Guibin Zhang, Yuqi Xu, Congcong Wang, Shuai Dong, Kaiwen Tuo, Xiangyu Zeng, Kaituo Feng, Qunzhong Wang, Yang Shi, Xiaobin Hu, Xiangyu Yue, Jiaqi Wang, Shuicheng Yan

llms efficiency multimodal

arXiv AI

Jun 30

Diagnosing and Mitigating Context Rot in Long-horizon Search

arXiv:2606. 29718v1 Announce Type: cross Abstract: Extensive context has become the norm as Large Language Models (LLMs) are increasingly deployed in long-horizon tasks.

By Shijie Xia, Yikun Wang, Zhen Huang, Pengfei Liu

llms efficiency benchmarks

arXiv AI

Jun 30

Spectral Perturbation of the Empirical Fisher Information Matrix under Weight Quantization

arXiv:2606. 28432v1 Announce Type: cross Abstract: We study the spectral perturbation of the empirical Fisher Information Matrix (FIM) of a parametric statistical model under two structured perturbations: departure of the input from a reference (in-distribution) ensemble, and finite-precision (quantized) perturbation of the model's parameters.

By Rahid Zahid Alekberli, Hikmat Karimov

llms efficiency

arXiv AI

Jun 30

Internalized Reasoning for Long-Context Visual Document Understanding

arXiv:2604. 02371v2 Announce Type: replace-cross Abstract: Visual long-document understanding is critical for enterprise, legal, and scientific applications, yet the best performing open recipes have not explored reasoning, a capability which has driven leaps in math and code performance.

By Austin Veselka

llms fine-tuning efficiency data-engineering

arXiv Machine Learning

Jun 30

Never Skip a Batch: Dense Learning of Temporal GNNs via Adaptive Pseudo-Supervision

arXiv:2505. 12526v2 Announce Type: replace Abstract: Temporal graph networks suffer from irregular supervision in realworld dynamic graphs, as most minibatches contain few labeled events.

By Alexander Panyshev, Dmitry Vinichenko, Oleg Travkin, Roman Alferov, Alexey Zaytsev

efficiency benchmarks

arXiv AI

Jun 30

Behavior Uncloning: Distilling Mode Redirection into Policy Weights without Inference-Time Steering

arXiv:2606. 29201v1 Announce Type: cross Abstract: Behavior-cloned policies often learn multiple behavior modes from demonstration datasets, including modes that are unsafe or otherwise undesired at deployment.

By Hao Wang, Jiuzhou Lei, Dayou Li, Bangya Liu, Minghui Zheng, Manling Li, Ruohan Zhang, Zhiwen Fan

diffusion robotics efficiency

arXiv AI

Jun 30

ConCise: Training-Free Conclusion-Chain State Compression for Cost-Efficient Multi-Step RAG Services

arXiv:2606. 28361v1 Announce Type: cross Abstract: Multi-step retrieval-augmented generation (RAG) has been widely deployed as LLM-powered web services for complex question answering, where iterative retrieval-reasoning rounds deliver strong multi-hop accuracy.

By Kuan Yan, Zhiqing Tang, Tian Wang, Weijia Jia

llms rag nlp efficiency

arXiv Machine Learning

Jun 30

Adaptive Iterative Hard Thresholding for Online High-dimensional Quantile Regression

arXiv:2606. 28652v1 Announce Type: cross Abstract: Online high-dimensional regression requires algorithms that can update sequentially while preserving structural sparsity.

By Zitian Zhou, Nan Lin

efficiency

Get the daily briefing