arXiv AI
ScalePRM: Training Process Reward Models by Scaling Verification Compute Without Ground Truth
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 30
Inducing Process Supervision from Outcome-Only Reinforcement Learning
arXiv:2609.36641v1 Announce Type: cross Abstract: Process reward models (PRMs) have become a key component for LLMs, as their step-level feedback supports both post-training and test-time reasoning....
arXiv AI
Aug 7
Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models
arXiv:2605. 12519v2 Announce Type: replace-cross Abstract: Training language models to produce both correct answers and sound reasoning remains an open challenge.
arXiv Machine Learning
Jun 9
The Hidden Bias of Process Reward Models:PRISM for Rewarding the Right Reasoning
arXiv:2606. 09078v1 Announce Type: new Abstract: Process Reward Models (PRMs) improve credit assignment for reasoning by providing step-level feedback.
arXiv AI
Oct 2
Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
The paper introduces POISE, a reinforcement learning algorithm that uses a model’s internal states as a value estimator to reduce variance in reinforcement learning with verifiable rewards (RLVR). By employing a lightweight probe that reads internal signals during the forward pass, POISE predicts baselines online and uses a cross‑rollout construction to keep gradients unbiased. Experiments on Qwen3‑4B and OLMo3‑7B‑Instruct‑DPO across six domains show POISE outperforms existing RLVR baselines, offering more stable training and a value model that generalizes across tasks and scales with the policy.
arXiv AI
Oct 15
Process Rewards for Outcome-Guided Reasoning Steps in LLM Reinforcement Learning
arXiv:2604.02341v2 Announce Type: replace-cross Abstract: Mathematical reasoning in large language models has improved substantially with reinforcement learning using verifiable rewards, where final...
arXiv AI
Jun 11
ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward
arXiv:2606. 11209v1 Announce Type: cross Abstract: Visual question answering increasingly requires multi-step reasoning.