arXiv Computation and Language
GRRM: Group Relative Reward Modeling for Machine Translation
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Aug 12
Reference-Free Post-Training of Open Large Language Models for Multilingual Machine Translation
arXiv:2608. 10812v1 Announce Type: cross Abstract: We study reference-free post-training for multilingual machine translation with open large language models.
arXiv Machine Learning
Sep 7
Consensus Group Relative Policy Optimization for Text Generation
Consensus Group Relative Policy Optimization (C‑GRPO) is a new training method that distills Minimum Bayes Risk (MBR) decoding into a group‑relative objective, enabling text generation models to approximate MBR performance without the costly inference‑time sampling and scoring. C‑GRPO only needs a utility function and policy samples, avoiding the need for gold references or curated preference data. Experiments on WMT 2024 machine translation and XSum summarization show that C‑GRPO matches MBR decoding quality while reducing inference overhead and outperforming other reference‑free baselines.
arXiv AI
2d ago
R3: Robust Rubric-Agnostic Reward Models
R3 is a new reward modeling framework that addresses limitations in current reward models by being rubric‑agnostic and generalizable across multiple evaluation dimensions. It provides interpretable, reasoned score assignments rather than opaque scalar outputs, enhancing transparency and flexibility in evaluating language models. The authors release their models, data, and code openly at https://github.com/rubricreward/r3.
arXiv AI
Jun 4
BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization
arXiv:2606. 04807v1 Announce Type: new Abstract: Mitigating social bias in Large Language Models (LLMs) presents a distinct alignment challenge: unlike verifiable tasks, bias lacks a single ground truth, creating a high-variance, subjective reward landscape.
Hugging Face Trending Papers
Jun 3
BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization
Mitigating social bias in Large Language Models (LLMs) presents a distinct alignment challenge: unlike verifiable tasks, bias lacks a single ground truth, creating a high-variance, subjective reward landscape. Previous preference-based fine-tuning methods have major trade-offs: Direct Preference Optimization (DPO) is limited by the lack of exploration inherent in offline training, while Proximal Policy Optimization (PPO) can lead to training instability due to potentially unreliable critic estimates.
arXiv Machine Learning
Aug 19
GUPO: Gradient Uncertainty-aware Policy Optimization for Post-Training Large Language Models
GUPO: Gradient Uncertainty-aware Policy Optimization for Post-Training Large Language Models proposes a new method for fine‑tuning LLMs after training. The approach models each group gradient as a random variable, estimates its probability distribution, and uses Dirichlet‑based gradient uncertainty to weight each group’s contribution during policy updates. Experiments on multiple benchmarks show that this uncertainty‑aware aggregation improves the effectiveness of post‑training policy optimization.