arXiv Computation and Language
RAWR: Reward Assignment Without Rollouts in Verifiable Domains
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Jun 11
ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward
arXiv:2606. 11209v1 Announce Type: cross Abstract: Visual question answering increasingly requires multi-step reasoning.
arXiv AI
3d ago
Inducing Process Supervision from Outcome-Only Reinforcement Learning
arXiv:2609.36641v1 Announce Type: cross Abstract: Process reward models (PRMs) have become a key component for LLMs, as their step-level feedback supports both post-training and test-time reasoning....
arXiv Machine Learning
Jun 9
The Hidden Bias of Process Reward Models:PRISM for Rewarding the Right Reasoning
arXiv:2606. 09078v1 Announce Type: new Abstract: Process Reward Models (PRMs) improve credit assignment for reasoning by providing step-level feedback.
arXiv AI
Jun 6
Controllable and Verifiable Process Data Synthesis for Process Reward Models
arXiv:2605. 02395v2 Announce Type: replace Abstract: Process reward models (PRMs) rely on high-quality process supervision data, yet existing construction methods often provide limited control over error location, error type, and trajectory consistency.
arXiv AI
Sep 21
LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solvers
arXiv:2609.21492v1 Announce Type: new Abstract: Chain-of-Thought (CoT) reasoning has been shown to improve the performance of large language models (LLMs), yet existing optimization methods largely r...
arXiv Machine Learning
Sep 4
Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning
The paper investigates whether the text of chain‑of‑thought reasoning steps actually reflects their true importance for a model’s final answer. By defining step importance as the advantage in expected reward when a step is included, the authors use Monte Carlo rollouts to estimate ground truth and then test whether large language model judges can identify high‑advantage steps. They find that capable LLMs can beat a prevalence baseline but still fall far short of a noise ceiling, and that fine‑tuning a step‑level critic improves detection for incorrect responses but remains distant from the ceiling for correct ones, indicating that step importance is only partially recoverable from the reasoning trace text.