arXiv:2606. 09078v1 Announce Type: new Abstract: Process Reward Models (PRMs) improve credit assignment for reasoning by providing step-level feedback.
arXiv:2606. 27739v1 Announce Type: new Abstract: Process reward models (PRMs) enhance the reasoning capabilities of large language models (LLMs) by providing fine-grained feedback, yet training PRMs typically requires expensive stepwise annotations.
arXiv:2609.36641v1 Announce Type: cross Abstract: Process reward models (PRMs) have become a key component for LLMs, as their step-level feedback supports both post-training and test-time reasoning....
arXiv:2606. 11209v1 Announce Type: cross Abstract: Visual question answering increasingly requires multi-step reasoning.
arXiv:2608.30051v1 Announce Type: new Abstract: Process reward models (PRMs) provide dense step-level guidance for search-based reasoning, enabling inference-time compute to be allocated toward promi...
Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult. Existing approache...
The paper introduces the Implicit Prefix-Value Reward Model (IPVRM), which learns the probability of eventual correctness for each prefix directly from outcome labels, thereby aligning training targets with inference-time step signals via temporal-difference differences. IPVRM improves step-verification F1 on ProcessBench. Additionally, the authors propose Distribution-Level RL (DistRL), a policy optimization method that applies TD advantages to both sampled and high-probability tokens, offering dense counterfactual updates without extra rollouts, and show that DistRL consistently enhances downstream reasoning when combined with IPVRM.
The paper introduces Stepwise Marginal Information Gain (MIG), an intrinsic process reward that evaluates how each reasoning step of a large language model (LLM) or vision-language model (VLM) improves the likelihood of the reference answer. MIG rewards only new likelihood maxima, preventing duplicate credit, and is combined with outcome, format, and self‑distillation objectives to guide training. Experiments on eight benchmarks show that this method outperforms outcome‑only reinforcement learning and improves accuracy by up to 4.8 points over binary‑reward training, including a 12.6‑point gain on MathVerse and a 12.9‑point advantage on vision‑language transfer at 7B parameters.
arXiv:2603.17815v2 Announce Type: replace Abstract: Understanding and evaluating multi-step reasoning in LLMs at the level of individual steps remains a key challenge. Process reward models (PRMs) pr...
arXiv:2608. 16156v1 Announce Type: new Abstract: Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult.
arXiv:2604. 09482v2 Announce Type: replace Abstract: Reasoning in knowledge-intensive domains remains challenging as intermediate steps are often not locally verifiable: unlike math or code, evaluating step correctness may require synthesizing clues across large external knowledge sources.
arXiv:2512.03244v2 Announce Type: replace-cross Abstract: Training process reward models (PRMs) requires step-level correctness labels, obtained either through expensive human annotation or by relyin...
arXiv:2605. 12519v2 Announce Type: replace-cross Abstract: Training language models to produce both correct answers and sound reasoning remains an open challenge.