arXiv Computation and Language
Small Language Models as Judges for Rubric-Based Reinforcement Learning
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
3d ago
Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation
Rubric-based evaluation of large language models (LLMs) often assigns binary judgments to multiple criteria, but the certainty of these judgments varies widely. Existing aggregation methods treat all criteria as equally certain, potentially misleading reinforcement learning signals. The authors propose Uncertainty-aware Rubric Weighting (URW), a training-free framework that uses conformal calibration to quantify and weight criterion-level uncertainty, improving preference accuracy and downstream RL alignment.
Hugging Face Trending Papers
5d ago
Rubrics from the Task, Verdicts from Evidence: Pointwise Rubric Rewards with Tool-Augmented Judges
The paper introduces ReVeR, a pointwise rubric reward model that derives hard rules and graded principles directly from instructions, avoiding response-induced bias. It employs tool-augmented judges and label‑blind distillation to create ReVeR‑Traces, a large dataset of rubric reasoning and judging trajectories. Experiments show ReVeR surpasses pairwise rubric models in preference accuracy, especially for instruction following, and reveal that higher preference accuracy does not always lead to better downstream policies.
arXiv AI
Oct 7
EnGRICH: Enhancing Generative Reward Modeling with Critiques from Humans
arXiv:2610.05370v2 Announce Type: replace Abstract: Generative reward models (GRMs) are important for LLM optimization. Unlike scalar reward models, GRMs generate natural-language critiques alongside...
arXiv Computation and Language
Sep 16
Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization
The paper introduces JudgeBiasBench, a benchmark that systematically quantifies judgment biases in large language model (LLM)-based judges across four dimensions and 12 bias types. It evaluates both generative and discriminative judges, revealing significant bias patterns that undermine reliability. The authors propose bias-aware training—reinforcement learning for generative judges and contrastive learning for discriminative judges—to reduce these biases while maintaining evaluation performance.
arXiv Machine Learning
Sep 22
RLVR$^{2}$: Reinforcement Learning with Verifiable Rubric-based Ranking
arXiv:2609.23457v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) is expanding from tasks with well-defined correctness signals, such as mathematics and code, towa...
arXiv AI
Jul 7
When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On
arXiv:2603. 05659v3 Announce Type: replace-cross Abstract: Reinforcement learning with verifiable rewards (RLVR) and Rubrics as Rewards (RaR) have driven strong gains in domains with clear correctness signals and even in subjective domains by synthesizing evaluation criteria from ideal reference answers.