arXiv AI By Juliette Decugis, Sean O'Brien, Francis Bach, Gabriel Synnaeve, Taco Cohen

Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RL

Read the original on arXiv AI →

arXiv:2607. 01490v1 Announce Type: cross Abstract: Reinforcement learning post-training dramatically improves LLM reasoning, but suffers from training instability and diversity collapse.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI
Jun 18

STARE: Surprisal-Guided Token-Level Advantage Reweighting for Policy Entropy Stability

arXiv:2606. 19236v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards algorithms like GRPO have emerged as the dominant post-training paradigm for complex reasoning in LLMs, yet commonly suffer from policy entropy collapse during training.

By Haipeng Luo, Qingfeng Sun, Songli Wu, Can Xu, Wenfeng Deng, Han Hu, Yansong Tang
arXiv AI
Aug 3

SAF-OPD: Stable Advantage Fusion for On-Policy Distillation

arXiv:2607. 29209v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) broadcasts a single response-level reward to every token, while on-policy distillation (OPD) scores each token against a stronger teacher for a dense advantage but caps performance at teacher quality and discourages exploration beyond it.

By Yifan Ding, Xincheng Wei, Yoshua Y. Li, Ziheng Li, Yuquan Lu, Siyu Zhang, Dongsheng Ma, Rongxiang Weng, Xunliang Cai, Yun Chen