Process Reward Modeling with Reliability- and Conservatism-Aware Bayesian Belief
Read the original on arXiv Machine Learning →Process Reward Models (PRMs) are crucial for scaling large language model reasoning at test time, but they suffer from two types of uncertainty: PRM uncertainty from limited supervision and imperfect training, and policy uncertainty from mismatched data collection policies. The paper introduces BayesianPRM, which treats process rewards as random variables, discretizes the reward space with an ensemble, and applies two belief updates—reliability‑aware updating for PRM uncertainty and conservatism‑aware calibration for policy uncertainty—to form a Bayesian belief whose expectation yields a more accurate reward estimate. Experiments on multimodal reasoning benchmarks show that BayesianPRM improves reward accuracy and test‑time scaling performance.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.