The paper introduces DEEPO, a Dual-Entropy Enhanced Policy Optimization method designed to mitigate hallucination in multimodal large language models (MLLMs). It addresses two weaknesses in reinforcement learning: (1) hard queries with high semantic entropy produce uniformly wrong samples, erasing advantage signals, and (2) confident-but-wrong tokens become invisible to gradients as the policy sharpens. DEEPO combines semantic‑entropy‑triggered expert prefixes to inject grounded continuations and Renyi preconditioning to counter logit saturation, yielding significant hallucination reduction while maintaining accuracy and training stability.
Reinforcement learning (RL) is widely used to sharpen reasoning in multimodal large language models (MLLMs), yet its effect on hallucination is uneven. We trace this to two weak points in the \emph{co...
The paper introduces Stable-MM-R1, a framework that stabilizes reinforcement learning for multimodal reasoning by addressing training instability and entropy collapse. It proposes Potential‑Aware Query Mining (PAQM) to filter data toward high‑potential samples and Hybrid Stratified Replay (HSR) to restructure batches using path entropy and reward stratification, reusing stability anchors and hard negatives. The method demonstrates superior performance on complex reasoning tasks compared to strong baselines.
arXiv:2606. 08815v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for eliciting long-chain reasoning in large language models.
arXiv:2607. 16205v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has emerged as a standard approach for enhancing reasoning in large language models, which typically optimizes the policy by contrasting multiple self generated rollouts.
arXiv:2604. 00860v3 Announce Type: replace Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a central post-training paradigm for improving the reasoning capabilities of large language models.
arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).
arXiv:2606. 19771v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced Large Language Model (LLM) reasoning; however, it faces a fundamental optimization instability: uniform token updates precipitate entropy collapse, leading to premature convergence to suboptimal strategies, whereas excessive Shannon Entropy maximization can cause entropy explosion, driving blind exploration toward incoherent reasoning chains.
arXiv:2609.37119v1 Announce Type: cross Abstract: Recent approaches to reinforcement learning (RL) post-training for large language models increasingly remove the critic to reduce training instabilit...
arXiv:2510. 01444v3 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards (RLVR) has advanced reasoning capabilities in multimodal large language models.
arXiv:2606. 31575v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a powerful tool for propelling Large Language Models (LLMs) beyond imitation-based training towards more robust reasoning capabilities.
arXiv:2608.28771v1 Announce Type: new Abstract: Large reasoning models achieve strong performance on complex tasks by generating extended chain-of-thought (CoT) traces via reinforcement learning with...
arXiv:2607. 06987v1 Announce Type: new Abstract: Reinforcement learning (RL) has become the standard paradigm for enhancing the complex reasoning capabilities of large language models (LLMs).