arXiv:2605. 26179v2 Announce Type: replace-cross Abstract: Density functional theory (DFT) serves as the basis for computational discovery in materials science and chemistry, yet each calculation demands extensive human effort: adjusting algorithms when convergence stalls, revising plans when unexpected physics emerges, and inserting steps as intermediate results reshape the problem.
The paper introduces a benchmark of 250 expert-curated questions on inertial confinement fusion to evaluate language models on scientific fluency, physical reasoning, and quantitative analysis. It compares various closed‑source, open‑source, and a specialized model called Prospero, showing that Prospero improves by 10.7 percentage points and that frontier models still make critical physics errors. Retrieval assistance helps smaller models but not frontier ones, and two stress tests highlight the need for expert assessment of physical assumptions.
arXiv:2607. 05199v1 Announce Type: new Abstract: Physics reasoning fails structurally in small language models: an error at any step propagates forward, corrupting every inference that follows.
arXiv:2512. 11935v2 Announce Type: replace Abstract: Agentic AI systems increasingly connect large language models (LLMs) to external scientific tools, yet whether and when tool access improves prediction accuracy remains uncharacterized.
arXiv:2607. 09789v1 Announce Type: new Abstract: We introduce PHITSBench, an execution-scored benchmark for the Monte Carlo Particle and Heavy Ion Transport code System (PHITS).
SCICONVBENCH is a benchmark designed to evaluate large language models (LLMs) on multi‑turn clarification tasks in computational science. It focuses on two key abilities: eliciting missing information (disambiguation) and resolving contradictory requests (inconsistency resolution) across four domains—fluid mechanics, solid mechanics, materials science, and partial differential equations. The benchmark pairs a structured task ontology with a rubric‑based evaluation framework, measuring LLM performance in clarification behavior, conversational grounding, and final‑specification fidelity, and reveals that even top models only resolve about 52.7% of disambiguation cases in fluid mechanics while often making ungrounded assumptions.
The paper introduces Code Consistency Preference Optimization Verification (CCPO), a method that generates computationally sound solutions with dependency graphs to improve execution-consistent preference optimization for language models. By building a scientific reasoning dataset and extracting reasoning steps, prerequisites, and derivability relationships, the authors compute execution consistency scores that are used to fine‑tune models such as Llama‑3‑8B and DeepSeekMath‑7B, achieving significant performance gains on MATH (+17.0%) and GSM8K (+15.1%). The extended Scientific Feasibility Control framework further boosts accuracy on PhyX physics reasoning to 50.1%, surpassing existing models while maintaining high scientific validity and reducing law violations.
LatticeMind is a validation‑gated, agentic framework that translates natural‑language requests into executable density functional theory (DFT) workflows. It uses workflow contracts to specify required stages, files, and dependencies, deterministic validators to check projects before solver launch, and bounded repair to fix failures, ensuring traceable reports linked to project files. On benchmarks spanning 44 RESCU and 19 VASP cases, LatticeMind reproduced all reference values, with 119 of 132 runs succeeding on the first validated plan and the rest after a single repair cycle, outperforming a conventional trial‑and‑repair baseline. whyItMatters":"The framework demonstrates that incorporating pre‑execution validation and repair can reliably prevent costly errors in multistage DFT workflows, making AI‑assisted scientific computing more trustworthy and efficient."
arXiv:2608. 03600v1 Announce Type: new Abstract: Partial differential equations (PDEs) become actionable in science and engineering not as isolated formulae, but as executable workflows that connect modelling assumptions, governing equations, numerical solvers, diagnostics, and decisions.
The paper investigates using Large Language Models (LLMs) as calibration agents for expensive engineering simulators by conditioning them on physics-labeled prompts that describe simulation errors. In tests on a cylinder-wake Large-Eddy Simulation and a damped-oscillator inverse problem, physics-informed LLM proposals outperform conventional derivative-free optimizers in early trials, but the advantage fades after fifteen evaluations. A hybrid workflow that uses the LLM as a warm-start followed by a conventional optimizer achieves target accuracy in only nine simulator runs, outperforming both the LLM alone and standard optimizers.
CompMat-Bench is a new benchmark comprising 94 tasks drawn from recent computational materials science studies, designed to evaluate AI agents on realistic research steps without requiring costly simulations during testing. The benchmark pre‑reproduces inputs and outputs to provide ground truth, allowing agents to be graded with fixed rules rather than an LLM judge. It supports both single tasks and multi‑step workflows, with varying levels of methodological guidance, and shows that while agents can achieve high pass rates on individual tasks, performance drops in longer workflows or with reduced guidance, often due to scientific rather than software errors.
arXiv:2607. 22596v1 Announce Type: new Abstract: Atomistic simulations are central to materials design, but their execution involves complex, multi-step workflows that require significant human expertise.