PRO-Step: Step-level Process Reward Optimization for Retrieval-Augmented Generation

PRO-Step introduces a step‑level process reward optimization framework for Retrieval‑Augmented Generation (RAG) that evaluates both logical validity and evidential grounding at each reasoning step. By training a generative Preference‑Based Reward Model (PRM) and using PRM‑guided value tree search to create preference pairs, the method optimizes the policy through step‑level Direct Preference Optimization. Experiments on single and multi‑hop QA benchmarks show that PRO‑STEP achieves the best average EM and F1 scores across five datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• rag
• benchmarks