arXiv AI
Jun 12
SciR: A Controllable Benchmark for Scientific Reasoning in LLMs
arXiv:2606. 13020v1 Announce Type: new Abstract: Three paradigmatic forms of inference recur across scientific reasoning: deduction, induction, and causal abduction.
SPARK: Skeleton-Guided Reasoning Synthesis from Large-Scale Scientific Literature
Related stories
arXiv AI
Aug 12
InfiniteScienceGym: An Unbounded, Procedurally-Generated Benchmark for Scientific Analysis
arXiv:2604. 13201v2 Announce Type: replace-cross Abstract: Large language models are emerging as scientific assistants, but evaluating their ability to reason from empirical data remains challenging.
arXiv AI
Jul 23
SciTrek: Evaluating and Improving Long-Context Numerical Reasoning over Scientific Articles
arXiv:2509. 21028v4 Announce Type: replace Abstract: We introduce SciTrek, a synthetic question-answering dataset for assessing and improving long-context numerical reasoning in large language models (LLMs).
arXiv AI
Aug 5
DocTrace: Towards Traceable Long Document VQA via Hierarchical Evidence Graph Reasoning
arXiv:2608. 03292v1 Announce Type: new Abstract: Long Document Visual Question Answering (LongDocVQA) requires Multimodal Large Language Models (MLLMs) to locate, integrate, and reason over heterogeneous document elements distributed across multiple pages.
arXiv AI
Jul 24
SciExplore: Evaluating Autonomous Agents from Scientific Navigation to Information Integration
arXiv:2607. 20926v1 Announce Type: new Abstract: Scientific research involves complex information-seeking and reasoning workflows across heterogeneous sources.
arXiv AI
Jun 9
DN-Hypo-Pipeline: An AI-Driven Workflow for Hypothesis Generation via Large Language Models and Scientific Explanations
arXiv:2606. 08532v1 Announce Type: new Abstract: A scientific hypothesis is the first step in research and undergoes experimental validation, yet it also reflects a deep understanding of and reasoning about scientific phenomena.
arXiv AI
6d ago
Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents
Sci‑MMR is a new benchmark for multi‑step evidence‑grounded scientific reasoning in multimodal agents, featuring 235 multi‑hop tasks across four disciplines and an average of nine figure panels per task. It evaluates not just final answer accuracy but also the recovery of structured evidence from scientific claims, citations, visual data, and supporting regions. Experiments on eight state‑of‑the‑art models show a gap of over 20 points between answer accuracy and complete evidence recovery, highlighting significant challenges in evidence acquisition and integration.
arXiv Computation and Language
Aug 28
Not Just Reason, Not Just Scan: Reinforcement Learning for Proactive Scientific Error Verification over Academic Paper
The paper introduces VERA-RL, a reinforcement‑learning framework for proactive scientific error verification in academic papers. It builds on a Reason–Verify–Scan workflow and presents VERA‑13K, a 12,900‑sample dataset with 4,300 matched reasoning chains covering six error categories across natural‑science domains. The authors also define fine‑grained rewards for reasoning completeness, evidence alignment, and error precision, and show that training Qwen3‑VL‑8B with VERA‑RL improves verifiable reasoning to levels comparable with flagship multimodal large language models.
arXiv AI
Jun 4
FALSIFYBENCH: Evaluating Inductive Reasoning in LLMs with Rule Discovery Games
arXiv:2606. 04751v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents in scientific tasks.
arXiv AI
Jul 28
Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers
arXiv:2509. 03059v2 Announce Type: replace-cross Abstract: Recent advances in Large Language Models (LLMs) have shown that their reasoning capabilities can be significantly improved through Reinforcement Learning with Verifiable Reward (RLVR), particularly in domains like mathematics and programming, where ground-truth correctness can be automatically evaluated.
arXiv AI
Aug 26
PARTAB: Partition-Aware Reasoning with Structured Evidence for Scalable Table Understanding
PARTAB is a framework that improves large language model reasoning on tables by constructing a structured evidence interface. It represents query‑relevant evidence as semantically coherent, row‑linked table regions and performs hierarchical selection over column groups and row‑level partitions before composing the evidence for answer generation. Evaluations on multiple table reasoning benchmarks show that PARTAB consistently outperforms full‑table prompting and recent methods, achieving strong performance on WikiTableQuestions and TabFact while remaining competitive on numerical reasoning tasks.
arXiv AI
Sep 7
SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding
SciDocBench is a workflow-centered benchmark for scientific document understanding that includes 124 expert-authored questions across seven capability groups and 19 subtasks in five scientific domains. Each question is evaluated under four conditions—English or Chinese, all-images-first or interleaved document representations—resulting in 496 evaluation instances. The benchmark is paired with SciDocIR, a typed evidence-graph representation, and SciDocDataset, a collection of 15K fine-tuning and 8K reinforcement-learning samples, forming an evaluation-to-training framework for scientific-document assistants.