BiCAA introduces bidirectional credit assignment for search-augmented agents by combining Forward Solvability Gain (FSG) and Subsequent Reasoning Gain (SRG). FSG measures how each search step increases the likelihood of the correct answer, while SRG evaluates how a step supports future reasoning, discouraging redundant retrieval. Experiments on seven open‑domain QA benchmarks and two Qwen backbones show that BiCAA improves average F1, especially on multi‑hop tasks.
Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult. Existing approache...
The paper introduces Grounded Credit Assignment (GCA) for reinforcement learning in multi‑turn search agents. GCA separates credit for evidence retrieval and answer correctness, correcting the bias in GRPO that favors evidence‑retrieving rollouts only proportionally to accuracy gains. Applying GCA to Search‑R1 boosts average EM by 5.7–6.3 points and increases the proportion of evidence‑supported correct answers from 76.9% to 93.4%.
arXiv:2608. 05102v1 Announce Type: new Abstract: Long-horizon search agents must make multiple sequential actions (steps) to search, retrieve, verify, and integrate evidence to reach a final answer.
arXiv:2608. 16156v1 Announce Type: new Abstract: Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult.
Search-G1 is a reward framework that encourages language agents to retrieve external information only when necessary and to ground their answers in that evidence. It uses two hidden‑state readouts—one predicting whether a closed‑book answer suffices and another measuring sensitivity to evidence deletion—to calibrate retrieval necessity and evidence reliance. The reward system gates evidence credit by necessity, rewards sufficient direct answers, and penalizes redundant searches, all without requiring costly annotations or LLM‑judge inference.
arXiv:2608. 07531v1 Announce Type: cross Abstract: Search-augmented language agents should retrieve external information only when necessary and ground their answers in retrieved evidence.
arXiv:2610.10179v1 Announce Type: cross Abstract: Search agents enable Large Language Models (LLMs) to iteratively retrieve and use information for complex multi-hop questions. Reinforcement Learning...
arXiv:2606. 00593v1 Announce Type: cross Abstract: Large language models are increasingly deployed as tool-augmented agents to acquire information beyond parametric knowledge.
TreeHCA introduces a method that couples hindsight-guided exploration with hindsight credit assignment for multi-turn large language model agents. By using changes in reference-answer likelihood to revisit intermediate histories and sample alternative continuations, it builds a tree of reasoning and tool interactions, and applies an inverse-hindsight importance correction to propagate terminal outcomes into turn-level advantages. Experiments on seven question-answering benchmarks with three Qwen backbones show that TreeHCA outperforms all baselines in aggregate exact match.
arXiv:2607. 13501v1 Announce Type: new Abstract: Reinforcement learning for multi-turn search reasoning typically relies on terminal outcome rewards, which cannot distinguish useful, redundant, and harmful intermediate interactions.
arXiv:2606. 10684v1 Announce Type: cross Abstract: Modern language agents which perform multi-step reasoning have shown strong performance in knowledge-intensive question answering.
Reinforcement learning for multi-turn search reasoning typically relies on terminal outcome rewards, which cannot distinguish useful, redundant, and harmful intermediate interactions. We propose LAPO, a self-generated process-supervision method based on backward leave-one-turn attribution.