SOAR (Source-Aware Reinforcement learning) is a new paradigm that transforms monolithic reasoning into source-explicit trajectories for search agents. It disentangles external evidence ($E_{ext}$) and internal knowledge ($E_{int}$) into parallel, observable streams via a Source-Aware Workflow and uses a Source-Consistent Reward to guide policy evolution. Evaluations show that SOAR improves performance on 3B and 7B backbones by 7.3% and 5.4% respectively, while reducing evidence fabrication by 74.8%.
arXiv:2604. 08477v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved reasoning in formal domains such as mathematics and code, but extending these gains beyond STEM remains challenging.
arXiv:2603. 12109v2 Announce Type: replace Abstract: Reinforcement learning (RL) has become a de facto paradigm for building LLM-based agents that act, interact, and reason over extended task horizons.
The paper introduces Proof-of-Use (PoU), a reinforcement learning framework designed to prevent tool‑call hacking in deep research agents. PoU enforces evidence grounding by requiring agents to cite normalized evidence identifiers and uses a multi‑objective reward system that includes progressive process rewards, an answer‑support alignment reward, and a curriculum‑style adaptive reward mixing. Experiments demonstrate that PoU effectively mitigates tool‑call hacking and promotes robust, adaptive tool usage even under domain and tool shifts.
KnowGPO introduces a reinforcement learning framework that groups search agent trajectories by the task-relevant knowledge they acquire, rather than by surface observations. By decomposing required information into atomic knowledge conditions and using the incremental knowledge gain as a process reward, KnowGPO provides finer-grained credit assignment for deep search tasks. Experiments on Xbench, BrowseComp-ZH, and GAIA demonstrate that KnowGPO outperforms outcome-only and process-supervision baselines by 3.8 to 4.2 points, with an 8B model rivaling or exceeding most 32B open-source systems.
arXiv:2608. 06128v1 Announce Type: new Abstract: Search agents extend large language models beyond static parametric memory by enabling them to acquire and use ex ternal evidence during multi-step reasoning.
IterSynth introduces a role-decoupled, iterative synthesis framework for deep search agents, separating planning and synthesis into distinct Planner and Synthesizer modules that maintain a persistent summary state. This design mitigates role coupling and context noise, while the new Role-Decoupled Policy Optimization (RDPO) enhances training by combining outcome rewards with turn-level rubric evaluations. Experiments on five long-horizon benchmarks show IterSynth-8B outperforming prior ≤8B agents by 4.2% and delivering significant zero-shot gains over ReAct on proprietary models.
arXiv:2606. 10684v1 Announce Type: cross Abstract: Modern language agents which perform multi-step reasoning have shown strong performance in knowledge-intensive question answering.
arXiv:2510. 13554v2 Announce Type: replace-cross Abstract: The reasoning pattern of Large language models (LLMs) remains opaque, and reinforcement learning (RL) typically applies uniform credit across an entire generation, blurring the distinction between pivotal and routine steps.
The paper introduces MIRA, a hierarchical architecture that separates meta-reasoning from execution for long-horizon research agents. An outer-loop meta-reasoner selects the next investigation based on a persistent research record, while an inner-loop executor carries out the chosen work order. The authors demonstrate that MIRA improves inference and compute allocation in theorem proving and neural-architecture research, and that a generative actor-critic variant (MIRA-AC) can be trained efficiently without directly optimizing long execution traces.
arXiv:2608. 12764v1 Announce Type: cross Abstract: Deep search agents operate over trajectories spanning dozens of steps, yet standard reinforcement learning provides only a single outcome reward per trajectory, which is far too sparse for effective credit assignment.
arXiv:2601. 07408v2 Announce Type: replace-cross Abstract: Group Relative Policy Optimization (GRPO) has emerged as a promising critic-free reinforcement learning paradigm for reasoning tasks.
arXiv:2603. 09803v2 Announce Type: replace Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in large language models but treats all correct solutions equally, potentially reinforcing flawed traces that arrive at correct answers by chance.