arXiv AI
Jul 21
Enhancing Rubric-based RL via Self-Distillation
arXiv:2607. 18082v1 Announce Type: cross Abstract: Rubric-based RL has recently shown promise in improving LLMs on open-ended tasks.
CriPO: Enhancing Rubric-based RL via Self-Distillation
arXiv:2607. 18082v3 Announce Type: replace Abstract: Rubric-based RL has recently shown promise in improving LLMs on open-ended tasks.
Related stories
arXiv AI
Jul 7
LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL
arXiv:2607. 04412v1 Announce Type: new Abstract: Reinforcement learning (RL) for non-verifiable instruction following increasingly relies on LLM judges with prompt-specific rubrics as reward signals.
arXiv Machine Learning
Aug 4
Verifier-Induced Support Reshaping in On-Policy Optimization
arXiv:2608. 00220v1 Announce Type: new Abstract: We show that on-policy reinforcement learning with verifiable rewards (RLVR) can improve the current objective while making successful behaviors for later objectives too rare to sample and reinforce.
arXiv AI
Aug 11
RISE-RL: Rubric-Informed Selective Exploration for Open-Ended Reinforcement Learning
arXiv:2608. 09123v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) for open-ended tasks is challenging because responses must satisfy multidimensional criteria without following a single correct generation trajectory.
arXiv AI
Aug 5
Rubrics as Privileged Information for Open-Ended Generation
arXiv:2608. 02948v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD), where a single model acts as both student and teacher with different contexts, has shown promise in verifiable domains like math, where hard privileged information (PI) in the form of ground-truth answers structurally constrains valid continuations.
arXiv AI
6d ago
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
arXiv:2608. 11669v1 Announce Type: cross Abstract: Reinforcement learning against rubrics, lists of criteria graded by an LLM judge, has become a standard way to post-train language models on tasks with no deterministic answer.
arXiv AI
Jun 2
CAST: Non-Privileged Clipped Asymmetric Self-Teaching with Advantage Flipping for GRPO
arXiv:2606. 00172v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR), especially Group Relative Policy Optimization (GRPO), has been widely used to improve reasoning in large language models.
arXiv AI
Jun 18
Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation
arXiv:2606. 19327v1 Announce Type: new Abstract: Post-training of reasoning language models is commonly driven by supervised distillation and reinforcement learning with verifiable rewards.
arXiv AI
Aug 6
Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation
arXiv:2608. 04794v1 Announce Type: new Abstract: Self-distillation (SD) has emerged as a compute-efficient alternative to reinforcement learning with verifiable rewards: a self-teacher, conditioned on privileged information (PI) about the answer such as a reference solution, supplies dense per-token supervision to a student that never sees it.
arXiv AI
Aug 7
DASH: Divergence-Adaptive Supervision Horizons for On-Policy Self-Distillation of Reasoning Models
arXiv:2608. 06243v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models using automatically verifiable outcome signals, but these signals are typically sparse and at the sequence-level.
arXiv AI
Aug 3
PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization
arXiv:2605. 17877v2 Announce Type: replace Abstract: A significant hurdle for current LLMs is the execution of complex, multi-stage tasks.
arXiv Machine Learning
Jun 18
Learning from Your Own Mistakes: Constructing Learnable Micro-Reflective Trajectories for Self-Distillation
arXiv:2606. 18844v1 Announce Type: new Abstract: Self-distillation improves reasoning in large language models by using the model's own rollouts as training signal, typically through implicit logit-level alignment that minimizes KL divergence toward a privileged target distribution.