Rubric-based evaluation of large language models (LLMs) often assigns binary judgments to multiple criteria, but the certainty of these judgments varies widely. Existing aggregation methods treat all criteria as equally certain, potentially misleading reinforcement learning signals. The authors propose Uncertainty-aware Rubric Weighting (URW), a training-free framework that uses conformal calibration to quantify and weight criterion-level uncertainty, improving preference accuracy and downstream RL alignment.
By Licheng Pan, Haocheng Yang, Haoxuan Li, Yuzhou Chen, Yichen Sun, Yuan Lu, Xiaoxi Li, Hao Wang, Zhixuan Chu, Kui Ren
arXiv:2608.30005v1 Announce Type: new
Abstract: Rubric-based reinforcement learning extends RL beyond tasks with exact answers or rule-based verifiers by scoring responses against instance-specific c...
By Fengyu Xie, Yilun Zhao, Bingsen Chen, Arman Cohan, Chen Zhao
arXiv:2608. 09123v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) for open-ended tasks is challenging because responses must satisfy multidimensional criteria without following a single correct generation trajectory.
By Jinkun Hou, Zhuo Liu, Huimin Ren, Hongsheng Xin, Pan Zhou, Kun Zhan
arXiv:2607. 20083v1 Announce Type: cross Abstract: Post-training with evaluator feedback on policy-induced samples serves as a major mechanism for improving large language models.
By Beining Wang, Weihang Su, Hongtao Tian, Hao Kong, Tao Yang, Ting Yao, Qingyi Pan, Yueyue Wu, Qingyao Ai, Min Zhang, Yiqun Liu
arXiv:2606.17029v2 Announce Type: replace
Abstract: Deep research agents synthesize long-form reports by searching and reasoning over retrieved evidence. Reinforcement learning with rubric-based rewa...
By Minghang Zhu, Chuyang Wei, Junhao Xu, Yilin Cheng, Zhumin Chen, Jiyan He
arXiv:2608.29856v1 Announce Type: new
Abstract: Large language models are increasingly used as scalable evaluators for open-ended tasks. However, many LLM judges derive query-specific criteria during...
By Yifan Chen, Haitao Li, Qingyao Ai, Fengbin Zhu, Tat-Seng Chua, Min Zhang, Yiqun Liu
arXiv:2607. 01830v1 Announce Type: new Abstract: Reliable reward and preference signals are critical for evaluating and optimizing large language models on open-ended tasks.
By Dazhi Fu, Jiuding Yang, Yiwen Guo, Jicong Fan
arXiv:2607. 06175v1 Announce Type: cross Abstract: Large language models (LLMs) can generate BPMN process models from natural-language descriptions, yet supervised fine-tuning (SFT) limits their output quality to the patterns present in the training data.
By Alexander Rombach, Chantale Lauer, Nijat Mehdiyev
EvoRubric introduces a co‑evolutionary reinforcement learning framework that learns rubric‑driven evaluation criteria for open‑ended text generation. The system uses a shared policy that simultaneously generates responses and rubrics, while a frozen meta‑verifier and grader provide validity checks and scoring. Experiments on medical, writing, and science benchmarks show that EvoRubric outperforms static and other evolving‑rubric baselines, achieving higher average scores on five tasks.
By Xin Guan, Xiaomeng Hu, Shen Huang, Zhenyi Wang, Bo Zhang, Zijian Li, Pengjun Xie, Bo Liu, Jiuxin Cao
R3 is a new reward modeling framework that addresses limitations in current reward models by being rubric‑agnostic and generalizable across multiple evaluation dimensions. It provides interpretable, reasoned score assignments rather than opaque scalar outputs, enhancing transparency and flexibility in evaluating language models. The authors release their models, data, and code openly at https://github.com/rubricreward/r3.
By David Anugraha, Zilu Tang, Lester James V. Miranda, Hanyang Zhao, Mohammad Rifqi Farhansyah, Garry Kuwanto, Derry Wijaya, Genta Indra Winata
arXiv:2608. 16156v1 Announce Type: new Abstract: Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult.
By Huan Zhang, Mingju Chen, Dongxu Zhou, Can Lv, Heng Chang, Sen Cui, Faguo Wu, Shiji Zhou
arXiv:2602. 10623v2 Announce Type: replace-cross Abstract: Reward models learned from human preferences are central to aligning large language models (LLMs) via reinforcement learning from human feedback, yet they are often vulnerable to reward hacking due to noisy annotations and systematic biases such as response length or style.
By Zhibin Duan, Guowei Rong, Zhuo Li, Bo Chen, Mingyuan Zhou, Dandan Guo