arXiv:2608. 03569v1 Announce Type: new Abstract: Benchmarking the ability of AI scientists to generate novel ideas is notoriously difficult.
arXiv:2609.07611v1 Announce Type: new Abstract: Scientific ideation is the capacity to formulate novel and testable hypotheses from scientific evidence, and autonomous AI scientists depend on it. Exi...
arXiv:2606. 11926v1 Announce Type: cross Abstract: Scientific progress depends on a repeated loop of exploration, experimentation, and abstraction.
Scientific progress depends on a repeated loop of exploration, experimentation, and abstraction. Researchers test candidate directions, interpret the evidence, and carry the resulting lessons into later attempts.
arXiv:2607. 06764v1 Announce Type: new Abstract: Recent progress on ARC-AGI-1 from disclosed architectures has come broadly from two regimes: heavy test-time compute over frontier models (evolutionary search, exhaustive sampling, extended chain-of-thought), or benchmark-specific training in which small models are fine-tuned on ARC data, often with task-specialized architectures.
The paper introduces agentic meta‑reasoning, a structured inference‑time framework that explicitly manages control decisions—such as selecting partial work, restarting, or stopping—during long‑horizon agentic tasks. By delegating task execution to workers and consolidating decisions through a lightweight controller that references persistent memory, the method reduces the need to replay full histories. Experiments on ProgramBench and other benchmarks show that meta‑reasoning improves performance over direct control baselines, especially as computation budgets increase, and reveals greater reuse of earlier work and higher solution coverage.
OpenDiscoveryTrace is a public dataset of 558 complete AI scientific agent trajectories that records the reasoning process—thoughts, tool calls, observations, errors, revision triggers, and confidence—across 124 scientific tasks in drug discovery, materials science, genomics, and literature analysis. The dataset includes seven models (three frontier models and four open‑weight models) and 60 live‑retrieval variants, providing a balanced view of performance and error patterns. Pilot analysis shows that process traces reveal behavioral differences invisible to output‑only evaluation, such as differing error rates and types among frontier models.
arXiv:2605. 10574v3 Announce Type: replace Abstract: As artificial intelligence advances, models are not improving uniformly.
ASI‑Bench is a new benchmark that evaluates AI systems on their ability to conduct innovative exploration and autonomous scientific research across 11 domains, using 60 project‑level tasks. It progressively removes human methodological guidance to test whether AI can independently select methods, execute research, and produce verifiable results. Results from 18 state‑of‑the‑art agent–model configurations show a sharp performance drop when guidance is reduced, indicating current systems still rely heavily on human input.
The paper introduces Agentic Reasoning for Tree Search (ARTS), a method that uses a reasoning language model to navigate the hypothesis‑experiment space in scientific discovery. Unlike traditional approaches that conflate hypothesis quality with execution quality and prune search logs, ARTS evaluates prior execution logs to distinguish implementation failures from poor hypotheses and selects the next hypothesis to pursue. By employing test‑time training to embed search‑tree knowledge into model weights, ARTS achieves a 15.3% relative improvement over leading algorithms on 22 benchmark tasks and enables smaller models like Qwen3‑4B to match or exceed the performance of larger closed‑source models at lower inference cost.
The paper "AI in Science: Early Insights" analyzes AI’s impact on scientific work using data from 15 million Gemini interactions, 2,600 specialized AI models, and a survey of 600 scientists. It finds widespread AI adoption, complementary use of large language models and specialized tools, significant productivity gains of about seven hours per week, and a shift in research bottlenecks toward hypothesis backlog and verification needs. The study suggests AI can boost scientific productivity but its full effect depends on addressing new downstream challenges.
arXiv:2607. 08758v1 Announce Type: new Abstract: Scientific ideas rarely start from a blank page.
arXiv:2606. 02863v1 Announce Type: new Abstract: AI-Driven Research Systems (ADRS) -- systems coupling LLMs with automated evaluation to discover algorithms, proofs, and designs -- are being optimized and adopted across domains, but the tools to analyze them have not kept pace.