arXiv AI
Center-of-Mass Interpretation of Evolution Strategies
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jul 1
Predictable GRPO: A Closed-Form Model of Training Dynamics
arXiv:2606. 30789v1 Announce Type: new Abstract: Group Relative Policy Optimization (GRPO) has become a standard tool for improving the reasoning ability of large language models, yet its training dynamics are still described empirically: reward trajectories are fit with low-parameter functional forms whose constants carry no mechanistic meaning, and hyperparameter choices remain a matter of trial and error.
arXiv Machine Learning
Oct 3
Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradients
The paper investigates why Group Relative Policy Optimization (GRPO) benefits from per‑prompt normalization by examining the local curvature of the sequence‑level policy gradient. It shows that standard deviation normalization acts as an adaptive gradient, yielding a provably faster convergence rate than unnormalized REINFORCE under mild conditions, with the improvement tied to the average within‑prompt reward standard deviation. The authors also propose IS‑GRPO, an importance‑sampling variant that maintains alignment with the full gradient and offers a tighter convergence guarantee, and empirically validate these theoretical insights on GSM8K and MATH datasets at 1.5B and 7B model scales.
arXiv Machine Learning
Aug 11
A Mean-Field Framework for Inference-Time Distributional Control of Diffusion Models
arXiv:2608. 08770v1 Announce Type: cross Abstract: Diffusion models are increasingly used as controllable samplers, whose generations can be steered at inference time according to a chosen reward function.
arXiv Machine Learning
3d ago
Reward Budgeting Reduces Premature Convergence in Reinforcement Learning for LLM Reasoning
arXiv:2611.05942v1 Announce Type: new Abstract: RL for LLM reasoning often reaches an early performance plateau as the policy distribution becomes increasingly sharp. Recent work has studied policy e...
Hugging Face Trending Papers
Aug 9
A Mean-Field Framework for Inference-Time Distributional Control of Diffusion Models
Diffusion models are increasingly used as controllable samplers, whose generations can be steered at inference time according to a chosen reward function. While such rewards are typically defined on individual samples, for many applications it is desirable to steer according to distribution-level rewards, for example to calibrate with population-level information or to encourage diversity.
arXiv Machine Learning
Oct 5
Understanding Enrichment in Reinforcement Learning
The paper investigates how enrichment—adding intermediate guidance to sparse rewards in reinforcement learning—affects policy-gradient updates. It shows that omitting or truncating importance-weight corrections implicitly reweights rewards, and decomposes the resulting gradient error into scale, rotation, and variance components. The authors propose a sequential Monte Carlo weight correction that mitigates variance growth, and apply their analysis to a fine-tuned Qwen3-1.7B model on OpenMathReasoning, illustrating how enrichment helps prevent collapse in sparse domains.