arXiv Machine Learning October 3, 2026 By Max Qiushi Lin, Jincheng Mei, Matin Aghaei, Michael Lu, Bo Dai, Alekh Agarwal, Dale Schuurmans, Csaba Szepesvari, Sharan Vaswani

Rethinking the Global Convergence of Softmax Policy Gradient with Linear Function Approximation: The Case of Multi-Armed Bandits

Read the original on arXiv Machine Learning →

arXiv:2505. 03155v2 Announce Type: replace Abstract: Policy gradient (PG) methods have played an essential role in the empirical successes of reinforcement learning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

reinforcement-learning

arXiv AI

Sep 24

Softmax gradient policy for variance minimization and risk-averse multi armed bandits

The paper introduces a new algorithm for the Multi‑Armed Bandit problem that prioritizes selecting the arm with the lowest variance rather than the highest expected reward, using a softmax policy parameterization. It constructs an unbiased estimate of the minimal‑variance objective by drawing two independent samples from the chosen arm and proves convergence under natural conditions. Numerical experiments demonstrate the algorithm’s practical behavior and provide implementation guidance, while also addressing general risk‑aware trade‑offs between average reward and variance.

By Gabriel Turinici

reinforcement-learning

arXiv Machine Learning

Aug 18

Last-Iterate Analyses of FTRL with the 1/2-Tsallis Entropy in Stochastic Bandits

arXiv:2510. 22819v3 Announce Type: replace Abstract: The convergence analysis of online learning algorithms is central to machine learning theory, where the last-iterate convergence is particularly important, as it captures the learner's actual decisions and describes the evolution of the learning process over time.

By Jingxin Zhan, Yuze Han, Zhihua Zhang

safety

arXiv Machine Learning

4d ago

Fine-Tuning on Self-Generated and Reward-Weighted Data: Learning Dynamics, Convergence Rates, and Benefits of Off-Policyness

arXiv:2609.36945v1 Announce Type: new Abstract: We study the learning dynamics of fine-tuning a policy model on self-generated and reward-weighted data, with particular focus on a generalized version...

By Zhiwei Wang, Yanxi Chen, Yaliang Li, Bolin Ding

llms reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 16

On the Sublinear Regret of Continuous K-Max Bandits

arXiv:2502. 13467v2 Announce Type: replace Abstract: The $K$-Max combinatorial multi-armed bandit problem arises in applications such as recommendation and distributed decision making, where the reward is determined by the maximum outcome among $K$ selected arms.

By Yu Chen, Siwei Wang, Longbo Huang, Wei Chen

reinforcement-learning safety

arXiv AI

Aug 14

Annealed Softmax Greedy in Many-Armed Bayesian Bandits

arXiv:2605. 31034v2 Announce Type: replace-cross Abstract: Reinforcement learning with verifiable rewards (RLVR) and group-based policy optimization methods such as GRPO update a stochastic policy by sampling multiple completions per prompt and increasing the policy's probability on those with higher reward, regularized by a KL penalty toward a reference policy.

By William Overman, Mohsen Bayati

reinforcement-learning

arXiv Machine Learning

Aug 3

Convergence and Regret of the Policy Gradient for Multi-Armed Bandits in Diffusion Environment

arXiv:2607. 29593v1 Announce Type: new Abstract: This paper studies the policy gradient update for a multi-arm bandit problem in diffusion environment that is described by a stochastic differential equation (SDE) under the continuous-time reinforcement learning framework by Wang et al.

By Yanwei Jia, Du Ouyang

diffusion reinforcement-learning

One email a morning, machine-written

Related stories

Softmax gradient policy for variance minimization and risk-averse multi armed bandits

Last-Iterate Analyses of FTRL with the 1/2-Tsallis Entropy in Stochastic Bandits

Fine-Tuning on Self-Generated and Reward-Weighted Data: Learning Dynamics, Convergence Rates, and Benefits of Off-Policyness

On the Sublinear Regret of Continuous K-Max Bandits

Annealed Softmax Greedy in Many-Armed Bayesian Bandits

Convergence and Regret of the Policy Gradient for Multi-Armed Bandits in Diffusion Environment