NIBench is a neuroimaging benchmark that evaluates autonomous AI agents on three fronts: execution and workflow reconstruction across 56 tasks in seven imaging modalities; skill evolution through distillation of execution trajectories; and scientific judgment via analysis of 36 Alzheimer’s disease claims. The study tests 19 language‑model agents, finding that top performers execute protocols nearly flawlessly, while distilled skills boost performance on known tasks but yield mixed results on unseen ones. Agents can reproduce reported results yet sometimes draw conclusions that diverge from expert answers, indicating that data access alone does not guarantee accurate scientific judgment.
arXiv:2605. 09366v3 Announce Type: replace Abstract: Transforming neuroimaging data into clinically actionable biomarkers is a knowledge-intensive and labor-intensive process.
arXiv:2607. 15079v1 Announce Type: new Abstract: Understanding the brain increasingly depends on integrating evidence across scales, modalities, and disciplines.
arXiv:2606. 07718v1 Announce Type: new Abstract: Agentic AI tools offer a promising path to automating software development bottlenecks in scientific research pipelines, particularly for stages that take domain experts days to months to build, where scientists care about correctness and robustness, not implementation details.
arXiv:2610.14381v1 Announce Type: new Abstract: Large language models are increasingly used as scientific assistants, yet we still lack reliable ways to evaluate whether they can investigate a system...
arXiv:2609.33357v2 Announce Type: replace Abstract: Reliable automated research requires agents to vet data, verify analyses, and generate hypotheses grounded in trustworthy evidence, potentially red...
arXiv:2606. 01961v1 Announce Type: new Abstract: Autonomous agents are increasingly expected to support end-to-end medical-AI research workflows, moving beyond isolated prediction tasks or short-form clinical question answering.
The paper introduces SciConBench, a large-scale benchmark with 9.21K health-related questions and expert-written conclusions, designed to evaluate AI agents’ ability to synthesize scientific conclusions. It employs an automated evaluation pipeline that breaks conclusions into atomic facts and measures factual precision and recall, and introduces SciConHarness, a clean‑room harness that limits web interaction to prevent data leakage. Experiments on eight frontier models and deep research agents show low factual quality (best F1 = 0.337) and that clean‑room settings consistently reduce performance, indicating that leakage inflates perceived capabilities. The study also audits consumer‑facing agents, finding they often produce incomplete or contradictory conclusions even when the correct answer is available.
arXiv:2609.39566v1 Announce Type: new Abstract: Foundation models can serve as clinical agents through tool-use harnesses. However, conventional medical benchmarks assess reasoning over preselected e...
Autonomous scientific research agents are increasingly applied to end-to-end scientific workflows, including literature review, data analysis, experimentation, and report generation. However, open-end...
Neuroscience data are fragmented across labs and formats, making reuse labor‑intensive. This study benchmarks coding agents on eight recent mouse neural recording papers, asking them to reformat data for a common decoder task. While agents handle many subtasks, they rarely produce fully error‑free solutions; the authors pair outcome metrics with a process rubric to evaluate errors and propose agent‑assisted data‑sharing practices.
arXiv:2409. 11363v2 Announce Type: replace-cross Abstract: AI agents have the potential to aid users on a variety of consequential tasks, including conducting scientific research.
NS-Copilot is a large‑language‑model driven multi‑agent system designed to automate neuroscience data analysis. It integrates domain‑specific pre‑trained models for modalities such as EEG and extracellular spike data, and uses a natural‑language interface to orchestrate agents that plan, generate code, and synthesize results. In benchmarks on Alzheimer’s, Parkinson’s, and working‑memory spike decoding, the system consistently outperformed strong baselines across multiple trials.