arXiv Machine Learning
Aug 4
CriPO: Enhancing Rubric-based RL via Self-Distillation
arXiv:2607. 18082v3 Announce Type: replace Abstract: Rubric-based RL has recently shown promise in improving LLMs on open-ended tasks.
Enhancing Rubric-based RL via Self-Distillation
arXiv:2607. 18082v1 Announce Type: cross Abstract: Rubric-based RL has recently shown promise in improving LLMs on open-ended tasks.
Related stories
arXiv AI
Sep 4
FlowBalance: Verifier-Grounded Self-Improvement from On-Policy Reasoning Experience
arXiv:2609. 03241v1 Announce Type: cross Abstract: A reasoning model can improve from its own on-policy experience, but this inner loop is fragile: terminal verifiers provide reliable yet sparse supervision, while dense same-model guidance can reinforce false confidence or overconcentrate learning on a narrow solution mode.
arXiv AI
1d ago
RISED: RubrIcs for agentic multi-environment Selection and sElf-Distillation
RISED introduces a framework that uses rubric-based textual feedback to improve training of a single large language model (LLM) agent across multiple interactive environments. By having an LLM judge tag rollouts with a shared rubric vocabulary, the system guides both online data selection and policy supervision, enabling richer cross‑environment relationships and within‑group reward contrast. Experiments show that RISED achieves the highest mean pass rate and ranks first or second in every individual environment, with rubric analysis revealing behavioural changes behind these gains.
arXiv AI
Aug 11
RISE-RL: Rubric-Informed Selective Exploration for Open-Ended Reinforcement Learning
arXiv:2608. 09123v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) for open-ended tasks is challenging because responses must satisfy multidimensional criteria without following a single correct generation trajectory.
Hugging Face Trending Papers
Aug 25
RecurSE: Bounded Recursive Self-Evaluation for LLM Rubric Judges
LLM-as-judge is essential for evaluating open-ended text and steering post-training, yet improving the judge itself typically relies on expensive annotations, reward models, or distillation from stron...
arXiv AI
Aug 5
Rubrics as Privileged Information for Open-Ended Generation
arXiv:2608. 02948v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD), where a single model acts as both student and teacher with different contexts, has shown promise in verifiable domains like math, where hard privileged information (PI) in the form of ground-truth answers structurally constrains valid continuations.
arXiv AI
Jul 7
LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL
arXiv:2607. 04412v1 Announce Type: new Abstract: Reinforcement learning (RL) for non-verifiable instruction following increasingly relies on LLM judges with prompt-specific rubrics as reward signals.
arXiv AI
Sep 2
CARE: Contrastive Anchor-based Rubric Evolution for Large Language Model Post-Training
arXiv:2609.00892v1 Announce Type: new Abstract: Rubric-based reinforcement learning decomposes open-ended instructions into prompt-specific, flexible rubrics, making it better suited than reinforceme...
arXiv AI
Sep 2
ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning
ARISE‑RL is a full‑cycle self‑evolution framework that couples a task/rubric generator with a reasoning solver, enabling open‑ended agents to learn via reinforcement learning without relying on gold answers. The generator creates tool‑grounded rubric criteria and rewards itself for producing valid, intermediate‑difficulty tasks, while the solver improves through fine‑grained rubric satisfaction signals and multi‑step reasoning. The framework also introduces Reward‑Gated Self‑Evolution Distillation to selectively distill memory‑augmented policies, reducing distribution mismatch, and it is evaluated on the new ECR‑Bench rubric suite, achieving state‑of‑the‑art performance across all benchmarks.
arXiv Machine Learning
Aug 4
Verifier-Induced Support Reshaping in On-Policy Optimization
arXiv:2608. 00220v1 Announce Type: new Abstract: We show that on-policy reinforcement learning with verifiable rewards (RLVR) can improve the current objective while making successful behaviors for later objectives too rare to sample and reinforce.
arXiv AI
3d ago
SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation
SIPO (Self‑Instructing Policy Optimization) unifies reinforcement learning with on‑policy self‑distillation by using a contrastive self‑teacher to generate token‑level credit signals. The method samples multiple rollouts per prompt, pairs each with a reference answer and its mistakes, and uses the difference in teacher log‑probabilities to provide dense feedback while still respecting the overall task reward. Experiments on reasoning and code‑generation benchmarks show that SIPO outperforms both RLVR and OPSD baselines without requiring an external teacher or extra generation steps.
arXiv AI
Aug 13
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
arXiv:2608. 11669v1 Announce Type: cross Abstract: Reinforcement learning against rubrics, lists of criteria graded by an LLM judge, has become a standard way to post-train language models on tasks with no deterministic answer.