Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

3,594 stories · RSS feed

arXiv Machine Learning
Jun 2

Single-Channel Tissue Segmentation via Cross-Modal Distillation from Foundation Models

arXiv:2606. 00928v1 Announce Type: cross Abstract: Multiplexed fluorescence microscopy improves tissue segmentation by providing complementary channels including nuclear (DAPI) and membrane (E-cadherin), that together encode richer spatial context than single-channel imaging alone.

By Sakib Mohammad, Jarin Ritu, Md Sakhawat Hossain
arXiv AI
Jun 2

Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillation

arXiv:2605. 09253v2 Announce Type: replace-cross Abstract: While recent work in Reinforcement Learning with Verifiable Rewards (RLVR) has shown that a small subset of critical tokens disproportionately drives reasoning gains, an analogous token-level understanding of On-Policy Distillation (OPD) remains largely unexplored.

By Yuxuan Jiang, Runchao Li, Shubhashis Roy Dipta, Dawei Li, Zhao Yang
arXiv AI
Jun 2

Mitigating Reward Hacking in RLHF via Bayesian Non-negative Reward Modeling

arXiv:2602. 10623v2 Announce Type: replace-cross Abstract: Reward models learned from human preferences are central to aligning large language models (LLMs) via reinforcement learning from human feedback, yet they are often vulnerable to reward hacking due to noisy annotations and systematic biases such as response length or style.

By Zhibin Duan, Guowei Rong, Zhuo Li, Bo Chen, Mingyuan Zhou, Dandan Guo
arXiv AI
Jun 2

Logit Distillation on Manifolds: Mapping by Learning

arXiv:2606. 00771v1 Announce Type: cross Abstract: A simple way to improve the performance of almost any machine learning model is not to train a single but several models with diverse algorithms which will make slightly distinct kinds of predictions and errors on the same data, and thus improve the average predictions and robustness.

By Yiru Yang, Junling Wang, Nishant Kumar Singh, Luohong Wu, Haoran Yan