Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

3,245 stories · RSS feed

arXiv Machine Learning

Jul 30

Global monitoring of methane point sources using deep learning on hyperspectral radiance measurements from EMIT

arXiv:2604. 10094v2 Announce Type: replace-cross Abstract: Anthropogenic methane (CH4) point sources are critical drivers of near-term climate forcing, safety hazards, and system-inefficiencies.

By Vishal V. Batchu, Michelangelo Conserva, Alex Wilson, Anna M. Michalak, Varun Gulshan, Philip G. Brodrick, Andrew K. Thorpe, Christopher V. Arsdale

llms computer-vision efficiency benchmarks safety

arXiv Machine Learning

Jul 30

Low-cost Embedded Breathing Rate Determination Using 802.15.4z IR-UWB Hardware for Remote Healthcare

arXiv:2504. 03772v3 Announce Type: replace-cross Abstract: Respiratory diseases account for a significant portion of global mortality.

By Anton Lambrecht, Stijn Luchie, Jaron Fontaine, Ben Van Herbruggen, Adnan Shahid, Eli De Poorter

efficiency

arXiv Machine Learning

Jul 30

Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities

arXiv:2505. 01043v2 Announce Type: replace Abstract: Large language models (LLMs) have achieved impressive performance across various domains.

By Zhiwei Hao, Jianyuan Guo, Li Shen, Yong Luo, Han Hu, Guoxia Wang, Dianhai Yu, Yonggang Wen, Dacheng Tao

llms efficiency

arXiv Machine Learning

Jul 30

Breaking the Curse with BAND: Nonparametric Distribution Estimation in High Dimensions

arXiv:2607. 26955v1 Announce Type: cross Abstract: Minimax-optimal rates for multivariate distribution estimation are known to suffer from the curse of dimensionality.

By Shuo-Chieh Huang, Chien-Ming Chi, Jau-er Chen

efficiency benchmarks

arXiv Machine Learning

Jul 30

Emergent Sparsity in Frozen Random CNN Feature Extractors for Deep Reinforcement Learning

arXiv:2607. 26059v1 Announce Type: new Abstract: We report a striking phenomenon: deep reinforcement learning agents trained with frozen, randomly initialized CNN feature extractors spontaneously develop extremely sparse fully-connected representations, without any sparsity-inducing objective.

By Scott M. Norton

agents reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jul 30

Automorphism-Induced Non-Canonicity in Top-k Explanations of Graph Neural Networks

arXiv:2607. 26344v1 Announce Type: new Abstract: A gradient-based GNN explainer given a molecule with two chemically equivalent nitro groups assigns them attribution scores that are equal to the last bit.

By Xin Xu, Siru Tao, Kaizhen Tan

efficiency

arXiv Machine Learning

Jul 30

Weak-to-Strong On-Policy Distillation

arXiv:2607. 26246v1 Announce Type: new Abstract: On-policy distillation (OPD), which aligns a student with the teacher's token-level distribution on the student's own rollouts, is an effective paradigm for transferring capabilities across LLMs.

By Fangxu Yu, Zinan Lin, Xiaodong Liu, Weijia Xu, Michael Xu, Tianyi Zhou, Jianfeng Gao

llms reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jul 30

The Sparsity Ceiling: Where Spiking Networks Can and Cannot Trade Activity for Energy

arXiv:2607. 26648v1 Announce Type: cross Abstract: Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates.

By Zeyu Wang

llms efficiency

arXiv Machine Learning

Jul 30

InferScale: GPU-Native KV Injection for Personalized LLM Serving

arXiv:2607. 27090v1 Announce Type: cross Abstract: Large language models are increasingly deployed with persistent personalized context, such as accumulated memory profiles or long conversation histories, that is shared across a user's many requests.

By Peter Li, Prashant Pandey

llms rag fine-tuning efficiency

arXiv Machine Learning

Jul 30

Kairos: Numerically Robust News Recommendation under Item Cold-Start via Cholesky-based LinUCB

arXiv:2607. 26832v1 Announce Type: new Abstract: Algorithmic news personalization in regional markets often fails because modern deep learning models require massive interaction data while real-world news has a short Time-to-Live (TTL < 48 h) and shallow article pools.

By Finn Hertsch

efficiency

arXiv Machine Learning

Jul 30

Teaching Tiny VLA Models Where to Look and How to Move

arXiv:2607. 04171v3 Announce Type: replace-cross Abstract: Tiny Vision-Language-Action models are appealing for real-time robotic control, but reducing model scale often weakens two capabilities essential for manipulation: task-conditioned spatial grounding and coherent action generation.

By Iok Tong Lei, Ying Jie Yap, Wei Huang, Qingchen Xie, Qianzhi Li, Yujie Zhang, Xiaolong Liu, Zhidong Deng

diffusion robotics efficiency multimodal

arXiv Machine Learning

Jul 30

When Top-K Misses the Decision: Tool-Call Drift in Multi-Teacher On-Policy Distillation

arXiv:2607. 07050v3 Announce Type: replace-cross Abstract: Top-K teacher logits make on-policy distillation tractable, but probability mass is not the same as decision support.

By Jiabin Shen, Guang Chen, Chengjun Mao

efficiency safety

arXiv Machine Learning

Jul 30

On-Policy Distillation for LLM Safety: A Routing Approach to Template-Robust Realignment

arXiv:2607. 27081v1 Announce Type: cross Abstract: Fine-tuning is the dominant paradigm for specializing large language models (LLMs), yet it exposes a critical vulnerability: malicious data providers can embed harmful behaviors into downstream corpora, creating models that retain professional skills while violating human values on demand.

By Yongjian Guo, Wanlun Ma, Lingyu Shen, Xi Xiao, Sheng Wen

llms fine-tuning efficiency benchmarks safety

arXiv Machine Learning

Jul 30

CoCaRS: Correlation Calibration-Based Redundancy Suppression for Heterogeneous Knowledge Distillation

arXiv:2607. 27054v1 Announce Type: new Abstract: Knowledge distillation (KD) enables a compact student model to learn from a powerful teacher and has become an effective paradigm for model compression.

By Fengming Yu, Haiwei Pan, Kejia Zhang, Chunling Chen, Jian Guan, Baoying Ma

efficiency

arXiv Machine Learning

Jul 30

FedTopo: Relation-Level Topology Sharing for Model-Heterogeneous Federated Learning

arXiv:2607. 26801v1 Announce Type: new Abstract: Federated learning (FL) enables collaborative learning over decentralized data silos without centralizing raw data.

By Zhaoyang Ma, Zhihao Wu, Xin Gao, Lipo Wang, Youfang Lin, Jing Wang

efficiency safety

arXiv Machine Learning

Jul 30

HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models

arXiv:2607. 26515v1 Announce Type: new Abstract: We present, to our knowledge, the first end-to-end FP4 RL post-training, in which both the rollout and training policies, including their forward and backward passes, operate at 4-bit precision.

By Hei Yi Mak, Shadan Golestan, Hoang Le, Mehran Taghian Jazi, Yunke Peng, Yaoyuan Wang, Yao Wang, Junsong Wang, Tianchi Hu, Fengchen He, Guipeng Hu, Tanzila Rahman, Anandharaju Durai Raju

llms reinforcement-learning efficiency

arXiv Machine Learning

Jul 30

RAG-HAR+: Towards Cost-Efficient LLM-Based Human Activity Recognition for Edge Deployment

arXiv:2607. 26631v1 Announce Type: new Abstract: Human Activity Recognition (HAR) from wearable sensors supports applications in healthcare, rehabilitation, fitness tracking, and smart environments.

By Hansi Karunarathna, Nirhoshan Sivaroopan, Chamara Madarasingha, Anura Jayasumana, Kanchana Thilakarathna

llms rag agents efficiency benchmarks

arXiv Machine Learning

Jul 30

Field Codes for Distributed Coupling Samplers and Certified Empirical Transport

arXiv:2607. 27078v1 Announce Type: cross Abstract: In this paper, we formulate three communication tasks for empirical optimal transport: distributed coupling sampling, cost-evaluable coupling output, and scalar value-certified sampling.

By Hung Mai, Hai Nguyen, Luong Doan, Ngoc Vu, Khanh Nguyen, Nhung Duong, Tuan Do

rag efficiency

arXiv Machine Learning

Jul 30

Lottery Tickets Are Not Deployment Tickets

arXiv:2607. 27031v1 Announce Type: new Abstract: Reports on how sparsification, compression, and lottery tickets change model behavior have been mixed in the prior literature, with beneficial effects observed in some studies and adverse effects in others.

By Bum Jun Kim

efficiency

arXiv Machine Learning

Jul 30

Lightweight Image Classification of Raptor Species for Edge Devices: Rare-Species Dataset Expansion via Video Frame Extraction, Knowledge Distillation, and TensorRT Deployment

arXiv:2607. 26238v1 Announce Type: cross Abstract: We investigate lightweight raptor-species classification for real-time edge deployment in wind-turbine collision mitigation.

By Takeshi Nishikawa

computer-vision fine-tuning efficiency

Get the daily briefing