← Back to all news
arXiv Machine Learning By Kaifei Wang, Zikang Shan, Liwei Wang, Yinyu Ye, Han Zhong

When Does Momentum Help in Reinforcement Learning with Verifiable Rewards?

Read the original on arXiv Machine Learning →

The paper investigates the role of first‑moment momentum in reinforcement learning with verifiable rewards (RLVR). Using a bandit formulation with bias‑corrected exponential moving average (EMA) momentum, the authors prove that, when policy gradients are exact, increasing the EMA coefficient β within a safe range accelerates optimization. In practice, they show that maintaining a larger β preserves a non‑vanishing expected gain even as the learning rate shrinks, and experiments on Qwen models confirm that higher β values improve validation accuracy by up to about 2 percentage points.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
2d ago

The Price of Current-Effect Inference in Nonstationary Multi-Armed Bandits

The paper studies the trade‑off between maximizing rewards and accurately estimating current treatment effects in nonstationary multi‑armed bandits. For $K$‑armed bandits with H"older‑smooth mean rewards, the authors derive minimax bounds on expected dynamic regret under a high‑probability simultaneous coverage requirement for all current effects. They propose the TRACE algorithm, which achieves a regret–precision trade‑off that separates reward‑learning regret from the unavoidable cost of refreshing current‑effect estimates, and extend the method with an empirical‑Bernstein e‑process to adapt exploration while preserving worst‑case guarantees.

By Sharan Sahu, Martin T. Wells
reinforcement-learning
More like this →
arXiv AI
Sep 2

Bandits in Prod: Hyperparameter Optimization at Inference Time

The paper introduces Online Hyperparameter Optimization (OHPO), framing it as an infinitely many‑armed bandit problem over mixed and conditional search spaces. It proposes the IMABO framework, which couples any bandit policy with any oracle for proposing new configurations, and presents IMOSS—a restart‑free anytime policy with provable regret bounds. Experiments show that IMABO, combined with practical oracles such as TPE, an incumbent‑mutation oracle, and a pretrained tabular foundation model, outperforms random search across a range of settings from classical ML models to LLM‑based agents.

By Louis Abraham, Tuan-Anh Nguyen, Nicolas Devatine
llmsagentsreinforcement-learning
More like this →