← Back to all news

Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

5,834 stories · RSS feed

arXiv Machine Learning
Sep 22

Efficient Mixture-of-Experts with Speculative Decoding via Expert Coactivation

The paper studies how the design of Mixture-of-Experts (MoE) routers affects inference speed when combined with Speculative Decoding (SD). It shows that routers promoting high expert coactivation reduce memory transfer costs and improve runtime. By integrating a global load‑balancing loss, shared experts, a consistency loss, and an autoregressive expert selection mechanism, the authors achieve a 21% throughput gain over baseline MoEs while preserving accuracy.

By Kumari Nishu, Han-Byul Kim, Santosh Chilkunda, Maxwell Horton, Arnav Kundu, Mohammad Samragh, Lauren Hannah, Mohammad Sekhavat, Nikhil Bhendawade, Manuel Ciosici, Iman Mirzadeh, Keivan Alizadeh Vahid, David Harrison, Irina Belousova, Mehrdad Farajtabar, Minsik Cho
llmsefficiency
More like this →
arXiv Machine Learning
Sep 22

Neural Residual Modeling for Scientific Data Compression under Guaranteed Error Bounds

The paper introduces a new compression pipeline for scientific simulation data that combines Residual Vector Quantization (RVQ) with a U‑Net post‑processing network to correct pixel‑space residuals, followed by a Guaranteed Autoencoder (GAE) that enforces block‑wise error bounds. The U‑Net is trained to predict spatially structured residuals, addressing limitations of latent‑space only approaches. Experiments on S3D, JHTDB, and E3SM datasets show improved NRMSE and compression ratios compared to RVQ alone while maintaining strict error guarantees.

By Surya Majumder, Liangji Zhu, Sanjay Ranka, Anand Rangarajan
diffusionefficiency
More like this →
arXiv Machine Learning
Sep 22

Learning Physics from an Imperfect Ancestor

arXiv:2609.24947v1 Announce Type: new Abstract: Neural operators evaluate parametric partial differential equations cheaply but degrade sharply outside their training distribution. Physics-informed n...

By S. Mohammad Mousavi, Teeratorn Kadeethum, Nikolaos Bouklas, Somdatta Goswami
efficiency
More like this →
arXiv Machine Learning
Sep 22

Spatial-Interactor: Learning Spatial Reasoning through Interaction with the Observable Physical World

arXiv:2609.23038v1 Announce Type: cross Abstract: Spatial reasoning is essential for vision-language models (VLMs) to understand and act in the physical world. Reasoning in dynamic environments requi...

By Kaixiang Yao, Xu Wang, Miao Pan, Hu Xiyue, Weishi Wang, Daniel Dahlmeier, Jintao Chen, Yongliang Shen, Xuhong Zhang, Wenqi Zhang
llmsfine-tuningefficiencymultimodalbenchmarks
More like this →