arXiv Machine Learning September 22, 2026 By Fanchao Chen, Ziheng Jiang, Ziyun Wei, Zheng Zhong, Du Li, Chi Zhang, Haibin Lin, Shivaram Venkataraman

Towards Full Pipeline FP8 Reinforcement Learning for LLMs

Read the original on arXiv Machine Learning →

The paper introduces Calibrated Clipping, a dynamic method to align FP8 quantization bounds with high‑precision BF16 distributions, thereby mitigating training instability in full‑pipeline FP8 reinforcement learning for large language models. It identifies that compounded FP8 noise distorts importance ratios, causing entropy surges and garbled outputs. Experiments across GRPO and DAPO algorithms on 8B‑32B models show the technique restores performance to BF16 levels.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 28

ACRL: Adaptive Control of Training-Inference Discrepancy for Stable Reinforcement Learning

arXiv:2607. 24062v1 Announce Type: cross Abstract: Reinforcement Learning (RL) training for Large Language Models (LLMs) often suffers from instability due to the discrepancy between training and inference.

By Wenwu Fan, Qihong Lin, Zhijie Xia, Zhuo Zheng, Sihao Wang, Qiang Chen, Liangsheng Zhu

llms reinforcement-learning efficiency

arXiv Machine Learning

Jul 20

QUADS: Stabilizing NVFP4 Reinforcement Learning for MoE via QUantization-error Alignment across Dual Sides

arXiv:2607. 15810v1 Announce Type: new Abstract: Rollout generation is a major bottleneck in Reinforcement Learning (RL) for Mixture-of-Experts (MoE) Large Language Models, motivating low-precision rollout acceleration such as FP8.

By Zhengyang Zhuge, Hao Yu, Xin Wang, Zheng Li, Yizhong Cao, Dayiheng Liu, Jianwei Zhang

llms reinforcement-learning efficiency benchmarks safety

arXiv Machine Learning

Jun 9

Rethinking the Divergence Regularization in LLM RL

arXiv:2606. 09821v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a key component of post-training large language models (LLMs).

By Jiarui Yao, Xiangxin Zhou, Penghui Qi, Wee Sun Lee, Liefeng Bo, Tianyu Pang

llms reinforcement-learning

arXiv Machine Learning

Jun 2

Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation

arXiv:2605. 21125v2 Announce Type: replace Abstract: Group Relative Policy Optimization (GRPO), a prominent algorithm within the Reinforcement Learning from Verifiable Rewards (RLVR) framework, has achieved strong results in improving the reasoning capabilities of large language models (LLMs).

By Xixiang He, Qiyao Sun, Ao Cheng, Xingming Li, Xuanyu Ji, Hailun Lu, Runke Huang, Qingyong Hu

llms reinforcement-learning benchmarks

arXiv AI

Jun 3

Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor

arXiv:2605. 20402v3 Announce Type: replace-cross Abstract: MXFP4 arithmetic can dramatically accelerate reinforcement learning (RL) post-training of large language models (LLMs), yet the quantization error introduces severe accuracy degradation.

By Xiaocan Li, Shiliang Wu, Zheng Shen

llms reinforcement-learning efficiency safety

arXiv Machine Learning

2d ago

Towards Better Training Signal: Advantage Clipped Policy Optimization

arXiv:2609.36816v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a cornerstone for improving the reasoning capabilities of large language models (LLMs), but the need for on-poli...

By Ruichuan Huang, Jinghan Liu, Congliang Chen

llms reinforcement-learning benchmarks

One email a morning, machine-written

Related stories

ACRL: Adaptive Control of Training-Inference Discrepancy for Stable Reinforcement Learning

QUADS: Stabilizing NVFP4 Reinforcement Learning for MoE via QUantization-error Alignment across Dual Sides

Rethinking the Divergence Regularization in LLM RL

Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation

Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor

Towards Better Training Signal: Advantage Clipped Policy Optimization