Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

6,536 stories · RSS feed

arXiv AI

Jul 7

FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion

arXiv:2602. 05305v3 Announce Type: replace-cross Abstract: Generating long-form content, such as minute-long videos and extended texts, is increasingly important for modern generative models.

By Zhuokun Chen, Jianfei Cai, Bohan Zhuang

llms diffusion efficiency

arXiv Machine Learning

Jul 7

FastCSP: Accelerated Molecular Crystal Structure Prediction with Universal Model for Atoms

arXiv:2508. 02641v2 Announce Type: replace-cross Abstract: Molecular crystal structure prediction (CSP) is essential for applications in pharmaceuticals and organic electronics.

By Vahe Gharakhanyan, Yi Yang, Luis Barroso-Luque, Daniel S. Levine, Sushree Jagriti Sahoo, Brandon M. Wood, Kyle Michel, Muhammed Shuaibi, Gregory J. O. Beran, Viachaslau Bernat, Misko Dzamba, Xiang Fu, Meng Gao, Xingyu Liu, Benjamin K. Miller, Keian Noori, Lafe J. Purvis, Tingling Rao, Ammar Rizvi, Matt Uyttendaele, Andrew J. Ouderkirk, Chiara Daraio, C. Lawrence Zitnick, Arman Boromand, Noa Marom, Zachary W. Ulissi, Anuroop Sriram

fine-tuning efficiency

arXiv Machine Learning

Jul 7

Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe

arXiv:2605. 03677v2 Announce Type: replace Abstract: On-policy distillation (OPD) has recently emerged as an effective post-training paradigm for consolidating the capabilities of specialized expert models into a single student model.

By Wenjin Hou, Shangpin Peng, Weinong Wang, Zheng Ruan, Yue Zhang, Zhenglin Zhou, Mingqi Gao, Yifei Chen, Kaiqi Wang, Hongming Yang, Chengquan Zhang, Zhuotao Tian, Han Hu, Yi Yang, Fei Wu, Hehe Fan

llms efficiency multimodal benchmarks

arXiv Machine Learning

Jul 7

Beyond the Need for Speed: Energy-Aware Code Generation via Simulation-Guided Reinforcement Learning

arXiv:2607. 04577v1 Announce Type: new Abstract: Code models strictly prioritize functional correctness, leaving software energy efficiency as an unoptimized byproduct.

By Saurabhsingh Rajput, Tushar Sharma

reinforcement-learning fine-tuning efficiency

arXiv AI

Jul 7

Do All Visual Tokens Matter Equally? Object-Evidence Preserving Token Merging for Vision-Language Retrieval

arXiv:2607. 04605v1 Announce Type: cross Abstract: Multi-vector vision-language retrieval preserves fine-grained visual evidence through maximum-similarity late interaction, but dense image-side tokens make storage and scoring expensive.

By Suhyeong Park, Junha Jung, Jungwoo Park, Jaewoo Kang

efficiency multimodal

arXiv AI

Jul 7

ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs

arXiv:2601. 07475v2 Announce Type: replace-cross Abstract: The emergence of fine-grained numerical formats like NVFP4 presents new opportunities for efficient Large Language Model (LLM) inference.

By Haoqian Meng, Yilun Luo, Yafei Zhao, Wenyuan Liu, Peng Zhang, Xindian Ma

llms efficiency benchmarks

arXiv Machine Learning

Jul 7

Learning When to Attend: Conditional Memory Access for Long-Context LLMs

arXiv:2603. 17484v2 Announce Type: replace-cross Abstract: Language models struggle to generalize beyond pretraining context lengths, limiting long-horizon reasoning and retrieval.

By Sakshi Choudhary, Aditya Chattopadhyay, Luca Zancato, Elvis Nunez, Matthew Trager, Wei Xia, Stefano Soatto

llms efficiency

arXiv Machine Learning

Jul 7

LGQ: Learnable Geometric Quantization for Image Tokenization

arXiv:2602. 16086v3 Announce Type: replace-cross Abstract: Recent collapse-free quantizers such as FSQ achieve stable training by replacing the learnable codebook with an engineered geometry: a fixed scalar grid whose structure is dictated by the codebook size K.

By Idil Bilge Altun, Mert Onur Cakiroglu, Elham Buxton, Mehmet Dalkilic, Hasan Kurban

diffusion nlp efficiency benchmarks

arXiv AI

Jul 7

Kwai Summary Attention Technical Report

arXiv:2604. 24432v2 Announce Type: replace-cross Abstract: Long-context ability, has become one of the most important iteration direction of next-generation Large Language Models, particularly in semantic understanding/reasoning, code agentic intelligence and recommendation system.

By Chenglong Chu, Guorui Zhou, Guowang Zhang, Han Li, Hao Peng, Hongtao Cheng, Hui Wang, Jian Liang, Jiangxia Cao, Kun Gai, Lingzhi Zhou, Lu Ren, Qi Zhang, Ruiming Tang, Ruitao Wang, Xinchen Luo, Yi Su, Zhiyuan Liang, Ziqi Wang, Boyang Ding, Chengru Song, Dunju Zang, Jiao Ou, Jiaxin Deng, Jijun Shi, Jinghao Zhang, Junmin Chen, Lejian Ren, Minxuan Lv, Qianqian Wang, Qigen Hu, Shiyao Wang, Siyang Mao, Tao Wang, Xingmei Wang, Zhixin Ling, Ziming Li, Zixing Zhang

llms rag agents efficiency

arXiv AI

Jul 7

Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving

arXiv:2602. 24044v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of adapters must be hosted concurrently.

By Ferran Agullo, Joan Oliveras, Chen Wang, Alberto Gutierrez-Torre, Olivier Tardieu, Alaa Youssef, Jordi Torres, Josep Ll. Berral

llms efficiency benchmarks

arXiv Machine Learning

Jul 7

Target-Guided Selective Reweighting for Physics-Informed Neural Network Inverse Problems: A Transfer Learning Approach

arXiv:2607. 05271v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) encounter ill-posed optimization, loss competition, and parameter compensation in partial differential equation (PDE) inverse problems.

By Qian Hu, Bin Fan, Yao Xiao, Zhicheng Lin, Meixin Xiong

diffusion fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 7

SAF3R: Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformers

arXiv:2607. 03612v1 Announce Type: cross Abstract: Feed-forward 3D reconstruction (F3R) transformers have recently achieved remarkable success.

By Jianing Deng, Yuanzhe Li, Jialu Wang, Song Wang, Tianlong Chen, Huanrui Yang, Jingtong Hu

llms computer-vision efficiency

arXiv Machine Learning

Jul 7

XS-VLA: Coupling Coarse-grained Spatial Distillation with Latent Flow Matching for Lightweight Robotic Control

arXiv:2607. 04171v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have shown strong multimodal understanding and spatial grounding, but their computational cost limits real-time robotic control.

By Lei Iok Tong, Qingchen Xie, Wei Huang, Ying Jie Yap, Yujie Zhang, Qianzhi Li, Xiaolong Liu, Zhidong Deng

llms diffusion robotics fine-tuning efficiency multimodal benchmarks

arXiv AI

Jul 7

Adaptive Inference Batching using Policy Gradients

arXiv:2607. 05272v1 Announce Type: cross Abstract: Inference serving systems must balance throughput and latency under bursty, heterogeneous workloads, yet the industry standard remains static batching policies that require manual tuning and cannot adapt to shifting traffic.

By Ruslan Sharifullin

agents reinforcement-learning efficiency

arXiv AI

Jul 7

TORINO: Token Reduction via Interpretable Concept Overlap in Vision-Language Models

arXiv:2607. 04593v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have demonstrated impressive capabilities across different tasks, but their computational cost is dominated by the large number of visual tokens fed to the language model.

By Riccardo Renzulli, Gabriele Spadaro, Shruthi Gowda, Alaa Eddine Mazouz, Van-Tam Nguyen

llms diffusion fine-tuning efficiency multimodal benchmarks

arXiv AI

Jul 7

TokAN: Accent Normalization Using Self-Supervised Speech Tokens

arXiv:2607. 03928v1 Announce Type: cross Abstract: Accent normalization (AN) seeks to convert non-native (L2) accented speech into standard (L1) speech while preserving speaker identity.

By Qibing Bai, Shuai Wang, Yuhan Du, Bohan Li, Yannan Wang, Haizhou Li

rag nlp reinforcement-learning fine-tuning efficiency

arXiv AI

Jul 7

HASSL: Hierarchy-Aware Self-Supervised Learning Framework for Single Cell Microscopy

arXiv:2607. 04353v1 Announce Type: cross Abstract: Hierarchical structure is common in image data, where fine-grained clusters often merge into larger, coarser semantic groups.

By Julius Riel, Vishwa Mohan Singh, Sai Anirudh Aryasomayajula, Anuun Chinbat, Hannes Leonhard, Moritz Ladenburger, Frederik Alexander, Vishisht Choudhary, Fabio Laredo, Giacomo Masserdotti, Thorben Prein, Carsten Marr, Amirhossein Kardoost

rag diffusion computer-vision efficiency

arXiv AI

Jul 7

Full-Stack FP4: Stable LLM Pretraining with Quantized Projections, Optimizers, and Attention

arXiv:2607. 04422v1 Announce Type: cross Abstract: Recent NVFP4 pretraining methods mainly target transformer linear layers, leaving optimizer states, optimizer arithmetic and attention underexplored in 4-bit pipelines.

By Siyu Ding, Mingchuan Ma, Jiabo Tong, Xingrun Xing, Ziming Wang, Guoqi Li

llms fine-tuning efficiency safety

arXiv Machine Learning

Jul 7

Selecting Samples on Graphs: A Unified Dataset Pruning Framework for Lossless Training Acceleration

arXiv:2606. 12913v2 Announce Type: replace Abstract: The rapid growth of modern training datasets has significantly increased computational cost, motivating dataset pruning~(DP) methods which retain only a subset of informative samples to reduce training cost.

By Dongyue Wu, Zilin Guo, Xiaoyu Li, Jiajia Liu, Jingdong Chen, Nong Sang, Changxin Gao

efficiency

arXiv Machine Learning

Jul 7

PhenoNEST: A Neuro-Symbolic Framework for Ontology-Aware Multimodal Plant Phenotyping and Trait Discovery

arXiv:2607. 03245v1 Announce Type: new Abstract: High-throughput plant phenotyping generates valuable data that often remains trapped in unstructured text and isolated RGB images.

By Jayant Ghadge, Soumyashree Kar, Surya S. Durbha

llms computer-vision efficiency multimodal

Get the daily briefing