arXiv AI

CARE: Contrastive Anchor-based Rubric Evolution for Large Language Model Post-Training

arXiv AI

Aug 13

Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL

arXiv:2608. 11669v1 Announce Type: cross Abstract: Reinforcement learning against rubrics, lists of criteria graded by an LLM judge, has become a standard way to post-train language models on tasks with no deterministic answer.

By Minglai Yang, Xinyu Guo, Utkarsh Tyagi, Mian Zhang, Razvan Dumitru, Sunjie Hou, Yunzhong He, Daniel Yue Zhang, Ying Liu

llms reinforcement-learning benchmarks safety

arXiv AI

Aug 17

Inducing Reward-Free Judging Rubrics that Reduce Over-Crediting in Agent Evaluation

arXiv:2608. 13564v1 Announce Type: new Abstract: Evaluating language-model agents at scale increasingly relies on a second language model as an automatic judge, because the gold signal, an executable environment reward, is expensive, slow, or unavailable at deployment time.

By Darragh Quinn, David Dylan, Roisin Healy, Fionn Carroll, Maeve Donnelly, Cormac Sheehan

llms agents fine-tuning

arXiv AI

Jul 21

Enhancing Rubric-based RL via Self-Distillation

arXiv:2607. 18082v1 Announce Type: cross Abstract: Rubric-based RL has recently shown promise in improving LLMs on open-ended tasks.

By Mingxuan Xia, Yuhang Yang, Chao Ye, Shuai Zhu, Shenzhi Yang, Guangcheng Zhu, Yuhang Zhang, Cheng Peng, Haobo Wang, Siqing Wang

llms reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Aug 4

CriPO: Enhancing Rubric-based RL via Self-Distillation

arXiv:2607. 18082v3 Announce Type: replace Abstract: Rubric-based RL has recently shown promise in improving LLMs on open-ended tasks.

By Mingxuan Xia, Yuhang Yang, Chao Ye, Shuai Zhu, Shenzhi Yang, Guangcheng Zhu, Yuhang Zhang, Cheng Peng, Haobo Wang, Siqing Wang

llms reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Oct 9

Evaluating Rubric Generation with Interventional Transfer

The paper introduces Interventional Transfer (IT), a method for evaluating rubric generation by assessing how two rubrics respond to perturbations that cause a pass/fail shift. IT reveals asymmetries in rubrics generated by different LLMs when applied to HealthBench, showing that improvements on one rubric do not reliably transfer to the other. The study contrasts IT with existing evaluation approaches, highlighting its ability to uncover these asymmetries.

By Erik Skalnes, Layne C. Price, Raviteja Anantha, Michael Oberst

llms benchmarks

arXiv Computation and Language

Sep 30

AdaTutoRank: Learning to Rerank Document Sets via Adaptive Tutoring Optimization for RAG and Deep Research

AdaTutoRank introduces a setwise document reranker that uses Adaptive Tutoring Optimization (ATO) to provide graded supervision across nine rubric dimensions. By generating hint‑based silver labels, reinforcement rewards, and distillation cues tailored to each rollout’s quality, the method improves credit assignment for individual documents within a set. Experiments on ten benchmarks covering Retrieval‑Augmented Generation (RAG), deep research, and setwise evaluation show that AdaTutoRank achieves superior overall performance while reducing the number of retrieval calls.

By Kailin Jiang, Lei Liu, Jian Xi, Yangqi Chen, Hui Xu, Hongwei Zhao, Bin Li, Yu Lu, Haibo Shi

rag reinforcement-learning efficiency benchmarks

arXiv AI

Aug 11

RISE-RL: Rubric-Informed Selective Exploration for Open-Ended Reinforcement Learning

arXiv:2608. 09123v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) for open-ended tasks is challenging because responses must satisfy multidimensional criteria without following a single correct generation trajectory.

By Jinkun Hou, Zhuo Liu, Huimin Ren, Hongsheng Xin, Pan Zhou, Kun Zhan

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 26

Reinforcement Learning without Ground-Truth Solutions can Improve LLMs

arXiv:2606. 27369v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) for training LLMs typically rely on ground-truth answers to assign rewards, limiting their applicability to tasks where the ground-truth solution is unknown.

By Yingyu Lin, Qiyue Gao, Nikki Lijing Kuang, Xunpeng Huang, Kun Zhou, Tongtong Liang, Zhewei Yao, Yi-An Ma, Yuxiong He

llms reinforcement-learning benchmarks

arXiv Machine Learning

Sep 22

RLVR$^{2}$: Reinforcement Learning with Verifiable Rubric-based Ranking

arXiv:2609.23457v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) is expanding from tasks with well-defined correctness signals, such as mathematics and code, towa...

By Hao Li, Zhengkun Zhang, Gangqiang Hu, Zhen Zhang, Yude Gao, Dai Dai, Jing Liu

reinforcement-learning benchmarks

arXiv AI

Sep 30

AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation

arXiv:2609.38142v1 Announce Type: new Abstract: A small trainable advisor can steer a frozen language-model executor using natural-language advice. In addition to learning from task rewards, the advi...

By Rishabh Agrawal, Hejie Cui, Shasha Li, Shanchan Wu, Sercan \"{O}. Ar{\i}k

llms reinforcement-learning efficiency

arXiv AI

1d ago

Projected Distillation: On Learning to Solve Hard Problems from Close, Correct Targets

arXiv:2610.27467v1 Announce Type: cross Abstract: On-policy learning is central to post-training: training on a model's own generations limits distribution shift and helps preserve existing capabilit...

By Shripad V. Deshmukh, Christian Bock, Kangrui Ruan, Abhijeet Awasthi, Vijay Lingam, Rein Houthooft, Manish Prajapat

agents reinforcement-learning efficiency

arXiv AI

Aug 18

TRCA: Transition-wise Rubric Credit Assignment for Long-horizon LLM Agents

arXiv:2608. 16156v1 Announce Type: new Abstract: Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult.

By Huan Zhang, Mingju Chen, Dongxu Zhou, Can Lv, Heng Chang, Sen Cui, Faguo Wu, Shiji Zhou

llms agents reinforcement-learning benchmarks

Related stories