arXiv Statistics ML
Entropy Regularization: A Free Correction to Cross-Entropy for Verified Demonstrations
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
Related stories
arXiv AI
Jul 1
Which Tokens Matter? Adaptive Token Selection for RLVR with the Relative Surprisal Index
arXiv:2606. 31575v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a powerful tool for propelling Large Language Models (LLMs) beyond imitation-based training towards more robust reasoning capabilities.
arXiv AI
Jun 4
Reinforcement Learning from Rich Feedback with Distributional DAgger
arXiv:2606. 05152v1 Announce Type: cross Abstract: Reasoning models have advanced rapidly, but the dominant reinforcement learning from verifiable rewards (RLVR) recipe remains surprisingly narrow: sample many responses and reward each with a single bit indicating whether the final answer is correct.
arXiv Machine Learning
Aug 11
Online Learnability of Chain-of-Thought Verifiers: Soundness and Completeness Trade-offs
arXiv:2603. 03538v4 Announce Type: replace Abstract: Large Language Models (LLMs) using chain-of-thought reasoning have demonstrated great potential for solving complex reasoning and planning tasks.
arXiv Computer Vision
Aug 27
Boosting Reasoning in Large Multimodal Models via Activation Replay
The paper introduces Activation Replay, a training‑free method that improves reasoning in post‑trained large multimodal models (LMMs) by replaying low‑entropy activations from the base model’s input context. It shows that Reinforcement Learning with Verifiable Rewards (RLVR) shifts low‑entropy activations and that modulating these activations enhances reasoning across tasks such as mathematics, visual agents, and video reasoning. Experiments demonstrate that Activation Replay outperforms alternatives like high‑entropy replay or direct cross‑model intervention, boosting Pass@K and broadening RLVR’s reasoning coverage.
arXiv AI
Jul 7
rePIRL: Learn PRM with Inverse RL for LLM Reasoning
arXiv:2602. 07832v3 Announce Type: replace-cross Abstract: Process rewards have been widely used in deep reinforcement learning to improve training efficiency, reduce variance, and prevent reward hacking.
arXiv Machine Learning
Jun 16
Entropy-Aware On-Policy Distillation of Language Models
arXiv:2603. 07079v3 Announce Type: replace Abstract: On-policy distillation is a promising approach for transferring knowledge between language models, where a student learns from dense token-level signals along its own trajectories.