arXiv AI
Jul 24
AREX: Towards a Recursively Self-Improving Agent for Deep Research
arXiv:2607. 21461v1 Announce Type: new Abstract: Deep research requires agents to find answers that jointly satisfy multiple constraints.
Learning What to Investigate Next: Meta-Reasoning for Long-Horizon Research Agents
The paper introduces MIRA, a hierarchical architecture that separates meta-reasoning from execution for long-horizon research agents. An outer-loop meta-reasoner selects the next investigation based on a persistent research record, while an inner-loop executor carries out the chosen work order. The authors demonstrate that MIRA improves inference and compute allocation in theorem proving and neural-architecture research, and that a generative actor-critic variant (MIRA-AC) can be trained efficiently without directly optimizing long execution traces.
Related stories
arXiv AI
Jun 11
Toward Generalist Autonomous Research via Hypothesis-Tree Refinement
arXiv:2606. 11926v1 Announce Type: cross Abstract: Scientific progress depends on a repeated loop of exploration, experimentation, and abstraction.
arXiv Machine Learning
1d ago
RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards
arXiv:2605.10899v2 Announce Type: replace-cross Abstract: Deep research agents plan, search, evaluate evidence, and synthesize long-form reports. Training these agents with reinforcement learning (RL...
Hugging Face Trending Papers
Jun 10
Toward Generalist Autonomous Research via Hypothesis-Tree Refinement
Scientific progress depends on a repeated loop of exploration, experimentation, and abstraction. Researchers test candidate directions, interpret the evidence, and carry the resulting lessons into later attempts.
arXiv AI
Jun 19
MetaResearcher: Scaling Deep Research via Self-Reflective Reinforcement Learning in Adversarial Virtual Environments
arXiv:2606. 19893v1 Announce Type: new Abstract: Deep research agents have demonstrated remarkable capabilities in autonomous information gathering and synthesis, yet their training remains constrained by the static nature of simulated environments, the limits of fact-retrieval-only task designs, and the inefficiency of outcome-based reinforcement learning.
arXiv AI
Sep 17
PrimeScientist: Strategic Allocation of Research Effort in Autonomous Research
PrimeScientist is a system that jointly selects research directions and allocates resources for autonomous research agents. It models the problem as a sequential decision task, using an executable plan tree to track competing plans and an adaptive MCTS-based policy to balance exploration and exploitation based on remaining resources and experimental feedback. Experiments on AI research, systems, code optimization, and machine learning engineering show that PrimeScientist improves average reward by 10.3% while reducing research attempts by 50.6% compared to AutoResearch under the same budget.
arXiv AI
Jul 23
In-the-Flow Agentic System Optimization for Effective Planning and Tool Use
arXiv:2510. 05592v2 Announce Type: replace Abstract: Outcome-driven reinforcement learning has advanced reasoning in large language models (LLMs), but prevailing tool-augmented approaches train a single, monolithic policy that interleaves thoughts and tool calls under full context; this scales poorly with long horizons and diverse tools and generalizes weakly to new scenarios.
Hugging Face Trending Papers
Jun 18
Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning
This work presents a general framework for training large language models (LLMs) to "Connect the Dots" (CoD), a meta-capability required by long-lifecycle agents: as an LLM-based AI agent gets deployed in an environment, it solves a long sequence of tasks while continuously exploring the environment, learning from its own experiences, and iteratively self-updating its context about the environment, thereby achieving progressively better performance on future tasks conditioned on the updated context. Major components of the CoD framework include: (1) algorithm design and infrastructure for end-to-end reinforcement learning (RL) with long rollout sequences interleaving solve-task and update-context episodes; (2) tasks and environments for incentivizing and eliciting the targeted meta-capability in LLMs during training, as well as for faithfully measuring progress during evaluation.
arXiv AI
Sep 30
Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning
The paper introduces agentic meta‑reasoning, a structured inference‑time framework that explicitly manages control decisions—such as selecting partial work, restarting, or stopping—during long‑horizon agentic tasks. By delegating task execution to workers and consolidating decisions through a lightweight controller that references persistent memory, the method reduces the need to replay full histories. Experiments on ProgramBench and other benchmarks show that meta‑reasoning improves performance over direct control baselines, especially as computation budgets increase, and reveals greater reuse of earlier work and higher solution coverage.
arXiv AI
Sep 30
MetaCtrl: Your Large Language Models Can Reason Better and More Concisely with a Metacognitive Controller
arXiv:2609.37304v1 Announce Type: new Abstract: Large reasoning models improve performance on challenging problems by allocating additional computation before answering, but longer reasoning does not...
arXiv AI
Jun 19
Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning
arXiv:2606. 20002v1 Announce Type: cross Abstract: This work presents a general framework for training large language models (LLMs) to "Connect the Dots" (CoD), a meta-capability required by long-lifecycle agents: as an LLM-based AI agent gets deployed in an environment, it solves a long sequence of tasks while continuously exploring the environment, learning from its own experiences, and iteratively self-updating its context about the environment, thereby achieving progressively better performance on future tasks conditioned on the updated context.
arXiv AI
Sep 25
IterSynth: Rethinking Deep Search Agents via Role-Decoupled Iterative Synthesis
IterSynth introduces a role-decoupled, iterative synthesis framework for deep search agents, separating planning and synthesis into distinct Planner and Synthesizer modules that maintain a persistent summary state. This design mitigates role coupling and context noise, while the new Role-Decoupled Policy Optimization (RDPO) enhances training by combining outcome rewards with turn-level rubric evaluations. Experiments on five long-horizon benchmarks show IterSynth-8B outperforming prior ≤8B agents by 4.2% and delivering significant zero-shot gains over ReAct on proprietary models.