arXiv AI
Expected Return Causes Outcome-Level Mode Collapse in Reinforcement Learning and How to Fix It with Inverse Probability Scaling
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Oct 15
MIPS: Mitigating Mode Collapse in Reinforcement Learning for Large Outcome Spaces
arXiv:2610.19457v1 Announce Type: new Abstract: Expected-reward maximization in reinforcement learning concentrates probability mass on a few outcomes, even in tasks that require many distinct, high-...
arXiv AI
Jun 2
Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying
arXiv:2606. 00151v1 Announce Type: cross Abstract: In reinforcement learning (RL), agents benefit from exploration only because they repeatedly encounter similar states: trying different actions can improve performance or reduce uncertainty; without such retries, a greedy policy is optimal.
arXiv AI
Jul 3
Trust Region Inverse Reinforcement Learning: Explicit Dual Ascent using Local Policy Updates
arXiv:2605. 11020v2 Announce Type: replace-cross Abstract: Inverse reinforcement learning (IRL) is typically formulated as maximizing entropy subject to matching the distribution of expert trajectories.
arXiv AI
Oct 1
Loop-Free Inverse Reinforcement Learning via Sequential Value Recovery with Q-Score Matching
arXiv:2609.38955v1 Announce Type: cross Abstract: Inverse Reinforcement Learning (IRL) aims to recover a reward function that explains expert demonstrations. Existing IRL methods typically rely on a...
arXiv AI
Sep 2
ReNFT: Repairing Mode Collapse in Reward Post-Training via Internal Probability-Mass Recalibration
ReNFT is a method that repairs mode collapse in diffusion generators after reward post‑training by internally recalibrating probability mass. It identifies suppressed alternatives through anti‑hub prompts and uses two policy‑dominated routes to generate counterfactual proposals, then applies reward‑based ranking and a joint‑and‑paired NFT update to restore diversity while preserving reward. Experiments on PickScore and GenEval show that ReNFT retains almost all of the original reward while significantly boosting diversity metrics.
arXiv AI
Aug 11
Parameter Exploration for RLVR via Variational Learning
arXiv:2608. 09805v1 Announce Type: cross Abstract: Exploration has been a focus of reinforcement learning research for a long time.