arXiv Computation and Language
A Survey on Rubric-Guided Reinforcement Learning for Language Models
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
2d ago
Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation
Rubric-based evaluation of large language models (LLMs) often assigns binary judgments to multiple criteria, but the certainty of these judgments varies widely. Existing aggregation methods treat all criteria as equally certain, potentially misleading reinforcement learning signals. The authors propose Uncertainty-aware Rubric Weighting (URW), a training-free framework that uses conformal calibration to quantify and weight criterion-level uncertainty, improving preference accuracy and downstream RL alignment.
arXiv Computation and Language
Sep 1
Small Language Models as Judges for Rubric-Based Reinforcement Learning
arXiv:2608.30005v1 Announce Type: new Abstract: Rubric-based reinforcement learning extends RL beyond tasks with exact answers or rule-based verifiers by scoring responses against instance-specific c...
arXiv AI
Aug 11
RISE-RL: Rubric-Informed Selective Exploration for Open-Ended Reinforcement Learning
arXiv:2608. 09123v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) for open-ended tasks is challenging because responses must satisfy multidimensional criteria without following a single correct generation trajectory.
arXiv AI
Jul 23
Co-Evolving LLM Evaluators and Policies via DynamicRubric
arXiv:2607. 20083v1 Announce Type: cross Abstract: Post-training with evaluator feedback on policy-induced samples serves as a major mechanism for improving large language models.
arXiv Computation and Language
3d ago
DEEPRUBRIC: Evidence-Tree Rubric Supervision for Efficient Reinforcement Learning of Deep Research Agents
arXiv:2606.17029v2 Announce Type: replace Abstract: Deep research agents synthesize long-form reports by searching and reasoning over retrieved evidence. Reinforcement learning with rubric-based rewa...
arXiv Computation and Language
Sep 1
GenRubric: Self-Evolving Rubric Generation for Scalable LLM Evaluation
arXiv:2608.29856v1 Announce Type: new Abstract: Large language models are increasingly used as scalable evaluators for open-ended tasks. However, many LLM judges derive query-specific criteria during...