PaperScout is an autonomous agent that treats academic paper search as a sequential decision-making process, dynamically deciding when and how to use search and expansion tools based on accumulated context. The authors identify a granularity mismatch in standard reinforcement learning for multi-turn tasks and propose Proximal Sequence Policy Optimization (PSPO), a sequence-level policy optimization method that aligns learning with agent–environment interactions. Experiments on synthetic and real-world benchmarks show that PaperScout outperforms structured retrieval and RL baselines in recall and relevance, demonstrating the effectiveness of its adaptive agentic framework and optimization strategy.
arXiv:2605. 11611v3 Announce Type: replace Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a promising paradigm for training agentic retrieval-augmented generation (RAG) systems from outcome-only supervision.
arXiv:2607. 24850v2 Announce Type: replace-cross Abstract: Recent advances in large language models (LLMs) have enabled search agents to autonomously tackle complex tasks across extended search and reasoning horizons.
Recent advances in Reinforcement Learning (RL) have substantially improved the capabilities of autonomous search agents, enabling sophisticated planning, and iterative retrieval over dynamic information sources. However, optimizing language models for specialized search behaviors often incurs an alignment tax, where gains in search performance come at the expense of general-purpose capabilities, limiting their effectiveness as universal assistants.
arXiv:2606. 02373v1 Announce Type: new Abstract: Search agents are often trained as policies over growing transcripts: the model must decide how to search while also remembering what it has seen, which evidence is useful, which constraints remain open, and which claims have actually been checked.
arXiv:2606. 20235v1 Announce Type: cross Abstract: Academic paper search is a core step in scientific research, and LLM-based search agents are emerging as a promising paradigm for iterative, intent-driven literature exploration.
arXiv:2606. 00593v1 Announce Type: cross Abstract: Large language models are increasingly deployed as tool-augmented agents to acquire information beyond parametric knowledge.
arXiv:2607. 21461v1 Announce Type: new Abstract: Deep research requires agents to find answers that jointly satisfy multiple constraints.
The paper introduces BRIDGE, a bilevel optimization framework that jointly trains a large language model (LLM) and a retriever for agentic reinforcement learning (ARL). It demonstrates that adapting the retriever before the policy yields better rewards, and that BRIDGE outperforms existing methods on seven open‑domain QA benchmarks and medical QA tasks, achieving significant gains in accuracy and reasoning quality.
arXiv:2605. 01248v3 Announce Type: replace Abstract: Reinforcement learning (RL) post-training has enabled newer capabilities in models, such as agentic tool-use for search.
Large language models (LLMs) are increasingly extended into deep search agents that solve complex questions through multi-step interaction with external search and browsing tools. However, existing agents often incur substantial computational and interaction costs, generating lengthy trajectories that contain redundant queries, inefficient exploration, and irrelevant observations.
The paper introduces a search‑aware reinforcement learning framework for multi‑component query understanding in Roblox game search. It first uses teacher‑student supervised fine‑tuning to create a schema‑compliant policy, then applies reinforcement learning that optimizes each query‑understanding component with component‑specific rewards derived from live search engine interactions. Experiments show that this approach improves per‑component utility and overall search quality, raising NDCG@20 by 8.9 points over the supervised baseline and 3.5 points over a single end‑to‑end reward strategy.
The paper introduces Conformalized Agentic Search (CAS), a framework that applies Conformal Prediction to improve the reliability of search agents during reinforcement learning fine-tuning. CAS uses an Adaptive Prediction Set (APS) to dynamically truncate retrieved documents based on statistical coverage, and Adaptive Conformal Inference (ACI) to construct confidence-aware prediction sets that penalize low‑confidence trajectories in the Group Relative Policy Optimization objective. Experiments on single‑hop and multi‑hop QA datasets show that CAS enhances reasoning accuracy and reduces redundant tool invocations, offering a more reliable and efficient agent paradigm.