GUPO: Gradient Uncertainty-aware Policy Optimization for Post-Training Large Language Models proposes a new method for fine‑tuning LLMs after training. The approach models each group gradient as a random variable, estimates its probability distribution, and uses Dirichlet‑based gradient uncertainty to weight each group’s contribution during policy updates. Experiments on multiple benchmarks show that this uncertainty‑aware aggregation improves the effectiveness of post‑training policy optimization.
arXiv:2606. 04807v1 Announce Type: new Abstract: Mitigating social bias in Large Language Models (LLMs) presents a distinct alignment challenge: unlike verifiable tasks, bias lacks a single ground truth, creating a high-variance, subjective reward landscape.
arXiv:2606. 08088v1 Announce Type: new Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has recently become a key paradigm for improving the reasoning abilities of Large Language Models (LLMs), yet it remains limited by sparse binary rewards and its ignorance of model-internal uncertainty.
arXiv:2606. 28707v1 Announce Type: new Abstract: Critic-free reinforcement learning with verifiable rewards (RLVR), exemplified by Group Relative Policy Optimization (GRPO), avoids training a value function (critic) and reduces memory and compute overhead relative to critic-based PPO pipelines for aligning large language models.
arXiv:2602.14028v2 Announce Type: replace Abstract: While Group Relative Policy Optimization (GRPO) offers a powerful framework for LLM post-training, its effectiveness in open-ended domains like Mac...
Consensus Group Relative Policy Optimization (C‑GRPO) is a new training method that distills Minimum Bayes Risk (MBR) decoding into a group‑relative objective, enabling text generation models to approximate MBR performance without the costly inference‑time sampling and scoring. C‑GRPO only needs a utility function and policy samples, avoiding the need for gold references or curated preference data. Experiments on WMT 2024 machine translation and XSum summarization show that C‑GRPO matches MBR decoding quality while reducing inference overhead and outperforming other reference‑free baselines.
arXiv:2607. 04242v1 Announce Type: new Abstract: Group-based reinforcement learning (RL) has become an effective paradigm for improving large language model agents on long-horizon interactive tasks.
Mitigating social bias in Large Language Models (LLMs) presents a distinct alignment challenge: unlike verifiable tasks, bias lacks a single ground truth, creating a high-variance, subjective reward landscape. Previous preference-based fine-tuning methods have major trade-offs: Direct Preference Optimization (DPO) is limited by the lack of exploration inherent in offline training, while Proximal Policy Optimization (PPO) can lead to training instability due to potentially unreliable critic estimates.
arXiv:2609.21432v1 Announce Type: new Abstract: Post-training plays a pivotal role in enhancing the reasoning capabilities and task-specific expertise of large language models (LLMs). Despite recent...
arXiv:2310.04363v3 Announce Type: replace Abstract: Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This limits t...
arXiv:2609.39321v1 Announce Type: cross Abstract: Group Relative Policy Optimization (GRPO) has emerged as a memory-efficient reinforcement fine-tuning (RFT) technique for reasoning-intensive tasks....
arXiv:2604. 00860v3 Announce Type: replace Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a central post-training paradigm for improving the reasoning capabilities of large language models.
arXiv:2605. 21125v2 Announce Type: replace Abstract: Group Relative Policy Optimization (GRPO), a prominent algorithm within the Reinforcement Learning from Verifiable Rewards (RLVR) framework, has achieved strong results in improving the reasoning capabilities of large language models (LLMs).