Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

6,338 stories · RSS feed

arXiv Machine Learning
Aug 12

Reinforcement Learning-based Semi-supervised Knowledge Distillation with LLM-as-a-Judge

arXiv:2604. 02621v2 Announce Type: replace-cross Abstract: Reinforcement Learning (RL) substantially improves the reasoning capabilities of language models, but most existing RL fine-tuning approaches rely entirely on ground-truth verifiable rewards and thus labeled datasets with verifiable answers.

By Yiyang Shen, Lifu Tu, Weiran Wang
arXiv AI
Aug 12

Exploration-Driven Personalized Federated Reinforcement Learning via Intrinsic Motivation

arXiv:2608. 10499v1 Announce Type: cross Abstract: Personalized Federated Reinforcement Learning (PFRL) takes a decentralized approach to storing and accessing information based on past experiences while keeping each client's data private during the learning of each client's policy.

By Md Rafid Islam, Rafsan Jany, Zahid Hasan, Ratun Rahman
arXiv Machine Learning
Aug 12

MoE Proxy Models for Low-Cost Failure Reproduction and Diagnosis in LLM RL Post-Training

arXiv:2608. 10823v1 Announce Type: new Abstract: Reinforcement learning (RL) post-training of large language models (LLMs) is computationally intensive and involves complex system pipelines with substantial debugging overhead.

By Yikai Wang, Chuansai Zhou, Yuhang Zhou, Weiqiang Wu, Cong Wu, Yue Deng, Ben Feng, Mingming Zhu, Beirong Zhou, Zhibin Wang, Sheng Zhong, Chen Tian, Wangze Zhang
arXiv Machine Learning
Aug 12

CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer

arXiv:2602. 11410v2 Announce Type: replace Abstract: Click-through rate (CTR) prediction is fundamental to online advertising systems.

By David Pardoe, Neil Daftary, Miro Furtado, Aditya Aiyer, Yu Wang, Liuqing Li, Tao Song, Lars Hertel, Young Jin Yun, Senthil Radhakrishnan, Zhiwei Wang, Tommy Li, Khai Tran, Ananth Nagarajan, Ali Naqvi, Yue Zhang, Renpeng Fang, Avi Romascanu, Arjun Kulothungun, Deepak Kumar, Praneeth Boda, Fedor Borisyuk, Ruoyan Wang
arXiv Machine Learning
Aug 12

HoloQ-VLA: Uniform W4A4 Quantization of Vision-Language-Action Models

arXiv:2605. 28803v3 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models unify perception, reasoning, and control in a single policy, but their multi-billion-parameter backbones and diffusion-based action heads make on-device deployment prohibitively expensive.

By Xinyu Wang, Mingze Li, Sicheng Lyu, Dongxiu Liu, Kaicheng Yang, Ziyu Zhao, Yufei Cui, Xiao-Wen Chang, Peng Lu