FARCA is a policy‑optimization framework that addresses the problem of noisy factual credit assignment in reinforcement learning for large language models. It transforms coarse factual supervision into token‑level training signals that are both localized and weighted by a reliability estimate derived from counterfactual evidence attribution. Experiments on several factual reasoning benchmarks show that FARCA improves model factuality while maintaining general reasoning abilities.
arXiv:2606. 27739v1 Announce Type: new Abstract: Process reward models (PRMs) enhance the reasoning capabilities of large language models (LLMs) by providing fine-grained feedback, yet training PRMs typically requires expensive stepwise annotations.
arXiv:2609.40360v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has improved the reasoning capabilities of large language models (LLMs), yet their predictions...
arXiv:2606. 15866v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become an effective post-training paradigm for improving the reasoning abilities of large language models.
arXiv:2606. 20068v1 Announce Type: new Abstract: While reinforcement learning from verifiable rewards (RLVR) typically has relied on a single binary verification signal, symbolic proof assistants in formal reasoning offer rich, fine-grained structured feedback.
The paper introduces Proof‑R1, a reinforcement‑learning framework that trains large language models to generate verifiable proofs for natural‑language logical reasoning tasks. Proof‑R1 only accepts a generated conclusion into the proof state when it satisfies formal verification constraints, ensuring each reasoning step is machine‑checkable. The method also reconstructs the dependency closure that supports the final answer, aligning credit with valid proof steps, and shows improved answer accuracy and verifiability across multiple benchmarks and models.
Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult. Existing approache...
arXiv:2607. 18258v1 Announce Type: new Abstract: Reinforcement learning from human feedback (RLHF) with preference-based reward models often exhibits unstable training dynamics.
arXiv:2606. 13262v1 Announce Type: new Abstract: Recent approaches combining Large Language Models (LLMs) with retrieval-augmented reasoning have shown promise for automated fact verification.
arXiv:2605. 12969v3 Announce Type: replace-cross Abstract: Group Relative Policy Optimization (GRPO) is one of the most widely adopted RLVR algorithms for post-training large language models on reasoning tasks.
arXiv:2606. 09078v1 Announce Type: new Abstract: Process Reward Models (PRMs) improve credit assignment for reasoning by providing step-level feedback.
arXiv:2609.00833v1 Announce Type: cross Abstract: Reinforcement learning (RL) for search agents typically relies on outcome rewards. However, it often fails to achieve effective credit assignment, du...
arXiv:2608. 16156v1 Announce Type: new Abstract: Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult.