Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

3,145 stories · RSS feed

arXiv Machine Learning
6d ago

RoutePack: Expert Placement and Attention-Aware Data Packing for MoE Reinforcement Learning

arXiv:2608. 12146v1 Announce Type: cross Abstract: Training Mixture-of-Experts (MoE) models for reinforcement learning (RL) couples two load-balancing problems: sequence composition determines dense attention work in each data-parallel microbatch, while token routing determines sparse expert work on expert-parallel ranks.

By Yibo Shen, Xudong Han, Xiaowei Zhu, Gen Li, Zhenxuan Pan
arXiv Machine Learning
6d ago

In Silico Study for Optimizing Intensity and Focality Electrode Configurations for Directional DBS Under Uncertainty Using Metaheuristic L1L1 Method

arXiv:2506. 13452v3 Announce Type: replace-cross Abstract: Background and Objective: As Deep Brain Stimulation (DBS) advances toward directional leads and optimization-based current steering, selecting electrode contact configurations becomes complex.

By Fernando Galaz Prieto, Antti Lassila, Maryam Samavaki, Sampsa Pursiainen
arXiv AI
6d ago

CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport

arXiv:2608. 13226v1 Announce Type: cross Abstract: While 3D Vision-Language Models (3D VLMs) have demonstrated remarkable spatial reasoning capabilities, they suffer from massive visual token counts that create severe computational bottlenecks during inference.

By Peng Ling, Yingda Yin, Lingting Zhu, Weikai Chen, Shengju Qian, Zeyu Hu, Xin Wang, Wenming Yang
arXiv AI
6d ago

Teach the Magnitude, Not the Direction: Verifier-Bounded Credit Assignment for Multi-Turn Multi-step LLM Agents

arXiv:2608. 13179v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) offers a verifier-bounded performance ceiling for training multi-turn tool-use agents, yet its trajectory-level credit assignment conflates heterogeneous per-turn outcomes into a single reward signal.

By Zechuan Wang, Siyuan Lu, Hongxuan Zhang, Linjian Mo, Chenyi Zhuang, Leilei Gan