Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

5,834 stories · RSS feed

arXiv Machine Learning

Sep 22

Taking a Second Look: Correcting Sea Ice Forecasts with Sparse Observations

arXiv:2609.24591v1 Announce Type: new Abstract: Sea ice forecasts are issued several days ahead, allowing errors to accumulate while new, often sparse sea ice concentration (SIC) observations become...

By Tianshuo Zhang, Xianglei Xing, Aowen Yang, Jia Gao, Wenzhe Zhai, ShanShan Liu

efficiency

arXiv Machine Learning

Sep 22

Learning Prognostic Variables for AI Convective Parameterizations via Symbolic Distillation

arXiv:2609.24882v1 Announce Type: new Abstract: Hybrid AI-physics climate modeling aims to improve coarse (~100km-resolution) Earth system models by learning to parameterize subgrid processes from hi...

By Jurij Sch\"onfeld, Tom Beucler, Julien Savre, Steven Sherwood, Veronika Eyring

diffusion efficiency

arXiv Machine Learning

Sep 22

AdaMem: Adaptive Memory Token Allocation for Soft Compression in Retrieval-Augmented Generation

arXiv:2609.22100v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) improves language models with retrieved evidence, but processing many long passages is costly and can introduce...

By Artem Sakhno, Grigorii Davydenko, Omar Zoloev, Julia Belikova, Andrey Savchenko, Maksim Makarenko

llms rag efficiency benchmarks

arXiv Machine Learning

Sep 22

Correlation-Aware Structured Pruning for Large Language Models

arXiv:2609.22131v1 Announce Type: cross Abstract: Structured pruning is a promising approach for reducing the substantial inference costs of Large Language Models (LLMs) while maintaining hardware ef...

By Sicheng Xu, Hao Shi, Wei Zhang, Haoran Pang, Zhenyu Ming, Hao Wu, Zhongyi Huang, Xin Yao, Gong Zhang

llms efficiency

arXiv Machine Learning

Sep 22

HFEMCNet: A Compact Hybrid Frequency Enriched Multi Channel Network for Automatic Modulation Classification

arXiv:2609.22139v1 Announce Type: cross Abstract: Automatic modulation classification (AMC) of received radio signals is prudent for further signal processing tasks such as communication monitoring,...

By Qamar Ijaz, Nayyer Aafaq

efficiency benchmarks

arXiv Machine Learning

Sep 22

Attention-Enhanced Dual-Branch ConvNeXt-BiLSTM Network for Subject-Independent EEG Seizure Detection

arXiv:2609.22141v1 Announce Type: cross Abstract: Automated seizure detection from scalp electroencephalography (EEG) is difficult because seizure morphology varies among patients and seizure samples...

By Maimuna Chowdhury, Sk. Imran Hossain

computer-vision efficiency benchmarks

arXiv Machine Learning

Sep 22

Beyond the Stitching Assumption: A Unified Framework for Multimodal Synthetic Data Evaluation via Semantic Quantization

arXiv:2609.22149v1 Announce Type: cross Abstract: Multimodal synthetic datasets combine structured attributes with free text, but are often evaluated separately. Such metrics can remain high after ta...

By Yefeng Yuan, Zhan Shi, Liang Cheng, Yuhong Liu

llms rag efficiency multimodal

arXiv Machine Learning

Sep 22

Mask-Aware Execution for Efficient JEPA Training

arXiv:2609.22674v1 Announce Type: cross Abstract: Joint Embedding Predictive Architectures (JEPAs) are becoming a core representation-learning primitive and a building block for latent world models a...

By Md Musfiqur Rahman Sanim, Zhihao Shu, Bahram Afsharmanesh, Amirali Mirian, Wei Niu, Gagan Agrawal

rag efficiency benchmarks

arXiv Machine Learning

Sep 22

Leveraging Industrial Foundation Models at the Edge of Particle Physics Detectors via Distillation Learning and Hardware Co-design

arXiv:2609.23385v1 Announce Type: cross Abstract: Data acquisition (DAQ) systems at future particle physics experiments stand to benefit from the extremes of AI/ML development: large-scale foundation...

By Gia Ancone, Qibin Liu, Liangyu Wu, Julia Gonski

fine-tuning efficiency

arXiv Machine Learning

Sep 22

PSD: Pseudo Self-Distillation of Memory Representation Capabilities for LLM Agents

arXiv:2609.23449v1 Announce Type: cross Abstract: Memory systems are becoming a core component of LLM agents, but constructing and maintaining memory remains expensive because it relies on repeated c...

By Pirzada Suhail, Menglin Xia, Xuchao Zhang, Mayukh Das, Chetan Bansal, Saravan Rajmohan

llms agents efficiency

arXiv Machine Learning

Sep 22

Sparse Regression Distilled from a Single Robust Fit

arXiv:2609.23937v1 Announce Type: cross Abstract: Robust linear fits can resist response contamination yet remain too dense or unstable for useful global explanations. We propose penalized distillati...

By Wooyoung Shin, Seunghwan Park

efficiency

arXiv Machine Learning

Sep 22

Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation

arXiv:2510.10925v3 Announce Type: replace Abstract: Training student models on synthetic data generated by strong teacher models is a promising way to distilling the capabilities of teachers. However...

By Hengyuan Zhang, Shiping Yang, Xiao Liang, Chenming Shang, Yuxuan Jiang, Chaofan Tao, Jing Xiong, Hayden Kwok-Hay So, Ruobing Xie, Angel X. Chang, Ngai Wong

efficiency