arXiv:2605.26087v2 Announce Type: replace-cross Abstract: Frontier LLMs now perform strongly across a wide range of physics evaluations, but it is hard to disentangle genuine reasoning from recall of...
arXiv:2607. 09195v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery.
The paper introduces an experimental model‑class revision framework that jointly proposes structural edits to a hypothesis space and diagnostic experiments to test those edits. By coupling a class‑level distinguishability objective with anytime‑valid sequential evidence, the method only revises the model class after the current one is rejected. On 400 controlled dynamical environments, the approach achieves 89.5% exact recovery with 32 experiments, outperforming baselines and transferring well to unseen mechanisms, library insufficiency detection, and other benchmark tasks.
arXiv:2604. 17406v3 Announce Type: replace Abstract: The convergence of large language models and agents is catalyzing a new era of scientific discovery: Agentic Science.
arXiv:2607. 05682v1 Announce Type: new Abstract: LLM systems for scientific discovery increasingly assist with ideation, literature synthesis, experiment planning, and report generation, but the first research question they propose can remain difficult to audit: it may sound plausible without exposing the mechanism, falsifier, or assumption that a scientist should inspect.
arXiv:2606. 04751v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents in scientific tasks.
arXiv:2607. 04293v1 Announce Type: cross Abstract: Building AI Scientist agents with Large Language Models (LLMs) has recently attracted growing attention.
The paper proposes a distributional theory explaining how large language models (LLMs) incorporate external evidence into their decision-making process. It identifies three key predictions: (1) evidence is more persuasive when it aligns with the model’s prior beliefs, (2) models more readily accept errors from their own internal processes than from external sources, and (3) the same evidence can improve weaker models while harming stronger ones. Extensive experiments across ten million trials, twelve LLMs from four families, and eight domains—including quantum mechanics, physics, genetics, and molecular biology—confirm these predictions and reveal that evidence integration occurs late in the network as a structured sequence of steps rather than through a simple trust metric.
ULTRADISCOVERY is a benchmark that tests abductive exploration in an epistemically open, interconnected universe. It features five domains where agents must revise an initial theory and predict outcomes of unseen cross‑domain interventions, with a $2 imes 2$ design manipulating representation openness and evidence alignment. Experiments show that agents struggle to compose evidence into transferable representations, especially when the representation is open, and that even with aids exact predictions are rare without large budgets.
The paper investigates whether large language models (LLMs) can perform principled Bayesian inference when updating beliefs in context. By creating an evaluation framework that includes qualitative epistemological tests, quantitative benchmarks, and analyses of downstream impacts, the authors find that LLMs typically violate Bayes' rule but that their performance improves dramatically when equipped with advanced reasoning and code‑execution tools, allowing them to approximate analytically intractable posteriors. The study suggests that appropriately tool‑augmented LLMs could already be leveraged to accelerate scientific discovery through complex belief updates.
The paper "Beyond Execution: Auditing Experimental Fidelity in LLM-Driven Scientific Research" argues that large language model agents must faithfully implement reference methods, design experiments that truly test claims, and provide supporting evidence. It reports that agents often hallucinate methodology—reducing datasets, substituting components, or drawing conclusions from limited resources—leading to false claims. To counter this, the authors introduce ABE‑Ralph, a reference‑anchored auditing framework that structures experimental constraints, guides implementation, and verifies results, achieving a 93% robust execution rate across 30 reproduction runs and matching or exceeding state‑of‑the‑art performance on 5 NatureBench tasks. "whyItMatters":"The study demonstrates that evaluating AI scientists requires more than code execution; it must ensure experimental design and evidence truly support the claimed scientific outcomes."
arXiv:2610.14381v1 Announce Type: new Abstract: Large language models are increasingly used as scientific assistants, yet we still lack reliable ways to evaluate whether they can investigate a system...