The paper introduces Research Attention Prediction (RAP), a benchmark designed to evaluate large language model (LLM) agents’ ability to infer changes across entire research fields from limited evidence gathered during online scholarly searches. RAP is built on a local search environment comprising 356,357 dated arXiv papers and includes two evaluation sets: a Core set of 2,160 questions covering 54 AI/ML fields and an Extended set spanning 269 fields. Experiments show that while search improves agents’ recognition of recent changes, their forecasting of future shifts remains limited or even negative, indicating that LLM agents can detect some recent developments but struggle to reliably anticipate future research activity.
arXiv:2609.10092v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly act as research agents, yet their ability to track shifts in research attention is difficult to evaluate be...
arXiv:2606. 00644v1 Announce Type: new Abstract: AI research often requires decisions before future evidence exists: which bottleneck to attack, which direction to pursue, or where a project should be positioned.
The paper proposes a framework for building autonomous AI research teams that can discover linear attention architectures. By assembling agents with diverse disciplinary expertise, private research histories, shared experimental evidence, and adaptive allocation of research opportunities, the team can balance exploration and exploitation under limited experimental budgets. Experiments show that this design outperforms GDN2 at 1.3B scale on language modeling, common‑sense reasoning, and retrieval tasks, and that removing any single component weakens discovery.
ScholarCatalyst is a new benchmark that evaluates how well AI systems can retrieve research papers that inspire new work. The dataset was created by having 184 lead authors of 207 recent computer science papers annotate which earlier papers helped their projects, providing detailed rationales. The benchmark tests retrieval from the literature available at the start of a project, revealing that current agentic search and even advanced models like Claude Fable 5.1 perform only modestly better than simple embedding retrieval.
JevGPT is a dual‑system architecture for autonomous research that combines fast evidence processing (System One) with deliberative research planning (System Two). System One quickly locates and evaluates relevant passages, while System Two sets evidence objectives and integrates findings into subsequent research steps. Experiments on question answering and document ranking demonstrate that JevGPT outperforms comparable agents, achieving faster completion, lower cost, and broader source coverage.
arXiv:2610.07625v1 Announce Type: cross Abstract: Automated research systems increasingly run LLM agents over long horizons, but more inference does not by itself produce more progress: agents replay...
arXiv:2609.13760v1 Announce Type: new Abstract: Publishing a research manuscript is a routine yet demanding part of scientific life: time-consuming, stressful, and often uncertain in outcome. Recent...
arXiv:2611.05720v1 Announce Type: new Abstract: Search can help LLM agents correct mistakes, but their starting conditions may shape which evidence they find and use. Drawing on cognitive-bias paradi...
arXiv:2609.14412v1 Announce Type: new Abstract: Deep research agents answer complex questions through iterative loops of searching, reading, and reasoning. Recent work on reasoning-intensive benchmar...
arXiv:2602. 20459v2 Announce Type: replace Abstract: Can AI systems trained on the existing scientific record forecast the advances that will follow?
IdeaTaste is a benchmark that evaluates large language models (LLMs) on their ability to generate research proposals that align with what human researchers actually publish. The study compares proposals generated by six contemporary LLMs against real papers from four research areas, finding that LLMs achieve only about 26% of the citation matching and 18% of the idea matching of human communities. Even larger LLM populations and richer field contexts do not close this gap, indicating that LLMs struggle more with translating research directions into concrete approaches than with selecting the right topics.
arXiv:2608.23045v1 Announce Type: new Abstract: Web search agents powered by Large Language Models (LLMs) show strong promise, but deep research tasks expose a recurring failure mode: once an agent h...