← Back to all news
arXiv Machine Learning By Weichu Xie, Haozhe Zhao, Wenpu Liu, Yongfu Zhu, Liang Chen, Minghao Ye, Zirong Chen, Yuqi Xu, Shuai Dong, Ziyue Wang, Xinbo Xu, Kean Shi, Ruoyu Wu, Xiaoying Zhang, Wenqi Shao, Baobao Chang, Nan Duan, Jiaqi Wang

Step-wise Rubric Rewards for LLM Reasoning

Read the original on arXiv Machine Learning →

The paper introduces Step-wise Rubrics as Rewards (SRaR), a reinforcement learning framework that assigns rubric-based scores to individual reasoning steps in large language models. By normalizing step scores and combining them with outcome rewards through a decoupled advantage estimator, SRaR preserves criterion-level structure and mitigates issues like rewarding flawed steps or encouraging endless self‑correction. Experiments on 16K problems and six math benchmarks show that SRaR outperforms prior rubric methods, improving accuracy, increasing faithful reasoning rates, and reducing self‑correction loops.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Jun 9

Correct Is Not Enough: Training Reasoning Planners with Executor-Grounded Rewards

arXiv:2605. 03862v4 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards has become a common way to improve explicit reasoning in large language models, but final-answer correctness alone does not reveal whether the reasoning trace is faithful, reliable, or useful to the model that consumes it.

By Tianyang Han, Hengyu Shi, Junjie Hu, Xu Yang, Zhiling Wang, Junhao Su
llmsreinforcement-learningbenchmarks
More like this →
arXiv AI
1d ago

Rubric Curriculum RL: Exploiting the Generation-Verification Gap in Non-Verifiable Domains

Rubric Curriculum RL (RcRL) is a reinforcement learning approach that leverages the generation‑verification gap in non‑verifiable domains by using pairwise preferences over rollouts and a curriculum over rubric criteria. Unlike traditional methods that rely on absolute scoring and stationary judges, RcRL provides a more discriminative, non‑stationary signal, allowing policies to maintain output entropy and continue improving beyond 1,000 training steps. In experiments on creative writing and HealthBench, RcRL outperforms baseline models, achieving a 70.5% human win rate and significant accuracy gains over the base model.

By Sumeet Ramesh Motwani, Tejas Krishnan, Alistair Letcher, Charles London, Suhaas Bhat, Huitian Jiao, Philip Torr, Riashat Islam, Christopher Summerfield, Jakob Foerster, Christian Schroeder de Witt, Qilong Gu, Shital Shah
reinforcement-learningbenchmarks
More like this →
arXiv Machine Learning
Sep 3

Cliff: Learning Process Rewards from the First Mistake

Cliff is a reward‑shaping method for reinforcement learning with verifiable rewards that identifies the first mistake in a language model’s reasoning process using an off‑the‑shelf teacher. By splitting each rollout into a correct prefix and an incorrect suffix, Cliff assigns token‑level advantages—positive for correct tokens and negative for the rest—providing fine‑grained supervision. Across 12 scenarios, Cliff outperforms on‑policy distillation by 15% and standard GRPO by 7%, even when the teacher is only modestly capable.

By Peixuan Han, Runhui Wang, Ketan Ramaneti, Jie Hao, Gerald Friedland, Chris Kong
llmsreinforcement-learningefficiency
More like this →
arXiv Computation and Language
4d ago

CRoS: Causality Reward on Scrambled CoTs Improves Accuracy and Reasoning Faithfulness

CRoS (Causality Reward on Scrambled CoTs) is a new causality metric for chain-of-thought reasoning that perturbs the order of reasoning steps while preserving their content, avoiding the confounding effects of traditional perturbations. On Reasoning Gym with Gemma‑3‑4B, CRoS improves out‑of‑domain accuracy (46.8% vs. 43.8%), yields higher CoT quality (average CoT‑Pass@K 43.4% vs. 41.4%), and enhances CoT faithfulness (hint acknowledgment 34.0% vs. 20.2%).

By Xiao Yu, Justin Chih-Yao Chen, Zaid Khan, Hyunji Lee, Mohit Bansal
llmsreinforcement-learning
More like this →