arXiv AI
Jun 12
SciR: A Controllable Benchmark for Scientific Reasoning in LLMs
arXiv:2606. 13020v1 Announce Type: new Abstract: Three paradigmatic forms of inference recur across scientific reasoning: deduction, induction, and causal abduction.
SCICONVBENCH: Benchmarking LLMs on Multi-Turn Clarification for Task Formulation in Computational Science
Related stories
arXiv AI
Jun 3
X-RAY: Mapping LLM Reasoning Capability via Formalized and Calibrated Probes
arXiv:2603. 05290v2 Announce Type: replace Abstract: Large language models (LLMs) achieve promising performance, yet their ability to reason remains poorly understood.
arXiv AI
Jul 9
MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoning
arXiv:2607. 07391v1 Announce Type: new Abstract: Mathematical reasoning benchmarks typically provide all facts needed to solve each problem, while interactive benchmarks often mix reasoning with tools, retrieval, and long-horizon dialogue.
arXiv AI
Aug 5
Large language models for partial differential equation workflows
arXiv:2608. 03600v1 Announce Type: new Abstract: Partial differential equations (PDEs) become actionable in science and engineering not as isolated formulae, but as executable workflows that connect modelling assumptions, governing equations, numerical solvers, diagnostics, and decisions.
arXiv AI
Jun 9
MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Science
arXiv:2510. 12171v2 Announce Type: replace Abstract: Large Language Models have shown strong scientific reasoning ability, but their performance on materials science problems remains less studied.
arXiv AI
Jul 24
SciExplore: Evaluating Autonomous Agents from Scientific Navigation to Information Integration
arXiv:2607. 20926v1 Announce Type: new Abstract: Scientific research involves complex information-seeking and reasoning workflows across heterogeneous sources.
arXiv Computation and Language
Sep 1
SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Tasks
arXiv:2606.15872v2 Announce Type: replace Abstract: Frontier scientific reasoning remains a major challenge for large language models (LLMs), where even the strongest commercial systems fall short of...
arXiv Machine Learning
Aug 27
SciMIF: Understanding Multimodal Instruction Following in Scientific Domains
SciMIF is a new benchmark that evaluates how well multimodal large language models (MLLMs) can follow complex scientific instructions. It is built on an analysis of 22 tasks across five scientific fields and introduces a taxonomy of 10 constraint groups that capture both general and discipline‑specific requirements. Experiments show large performance gaps between fields—chemistry is hardest—and that larger models do not necessarily improve constraint adherence, especially for fine‑grained, knowledge‑heavy instructions.
arXiv AI
Jul 23
SciTrek: Evaluating and Improving Long-Context Numerical Reasoning over Scientific Articles
arXiv:2509. 21028v4 Announce Type: replace Abstract: We introduce SciTrek, a synthetic question-answering dataset for assessing and improving long-context numerical reasoning in large language models (LLMs).
arXiv AI
Jul 28
From Execution to Capability: Scientific Experience Consolidation via Procedural Knowledge Synthesis
arXiv:2607. 24459v1 Announce Type: new Abstract: Large language models increasingly solve scientific-computing tasks, but executable feedback from one problem rarely becomes durable capability on subsequent problems.
arXiv AI
4d ago
ZEBRAARENA: A Diagnostic Simulation Environment for Studying Reasoning-Action Coupling in Tool-Augmented LLMs
arXiv:2603.18614v2 Announce Type: replace Abstract: Tool-augmented large language models (LLMs) must tightly couple multi-step reasoning with external actions, yet existing benchmarks often confound...
arXiv AI
Jul 28
Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers
arXiv:2509. 03059v2 Announce Type: replace-cross Abstract: Recent advances in Large Language Models (LLMs) have shown that their reasoning capabilities can be significantly improved through Reinforcement Learning with Verifiable Reward (RLVR), particularly in domains like mathematics and programming, where ground-truth correctness can be automatically evaluated.