SCAFFOLD is a large-scale structured dataset of computer science research figures, each paired with captions, context, questions, answers, and chain-of-thought reasoning traces. It contains 157,387 figure–question pairs from 3,058 arXiv papers, with subsets of 36,797 and 12,000 pairs for medium and small-scale use. The dataset was created using layout detection, PDF parsing, and AI-assisted question generation, and was used to benchmark a vision‑language model (Qwen2.5‑VL‑3B‑Instruct).
arXiv:2610.14173v1 Announce Type: cross Abstract: Digital libraries increasingly host scientific publications whose figures encode primary research findings, yet making this visual content searchable...
VELD-QA is a large-scale benchmark for visual evidence grounding and question answering in long documents, comprising 15,000 human-authored question-answer pairs across 5,859 PDFs with 372,445 pages. The dataset includes detailed annotations of supporting pages, visual evidence elements, and descriptions of cited visual content, enabling separate evaluation of retrieval, localization, and interpretation. Experiments on nine vision-language models and various retrieval-augmented systems reveal significant gaps in evidence localization and visual interpretation, with oracle accuracy peaking at 81.7% and enrichment of visual-element annotations narrowing performance gaps by 17–24%.
The paper introduces SciGram, a large-scale dataset of 194K scientific diagrams paired with 1.4M visual instructions generated through a terminology‑grounded pipeline that extracts domain concepts, synthesizes facts, and retrieves relevant diagrams. Models fine‑tuned on SciGram show significant gains on diagram‑centric benchmarks such as TQA, ScienceQA, and AI2D, and when combined with existing models like LLaVA OneVision, set new state‑of‑the‑art performance. The authors release both the dataset and trained models to support further research in scientific diagram understanding.
SciDocBench is a workflow-centered benchmark for scientific document understanding that includes 124 expert-authored questions across seven capability groups and 19 subtasks in five scientific domains. Each question is evaluated under four conditions—English or Chinese, all-images-first or interleaved document representations—resulting in 496 evaluation instances. The benchmark is paired with SciDocIR, a typed evidence-graph representation, and SciDocDataset, a collection of 15K fine-tuning and 8K reinforcement-learning samples, forming an evaluation-to-training framework for scientific-document assistants.
arXiv:2606. 28406v1 Announce Type: new Abstract: Text-to-image and multimodal generative models are increasingly used to produce scientific figures such as mechanism diagrams, experimental-design schematics, conceptual frameworks, and graphical abstracts.
While Text-to-Image (T2I) models have shown remarkable success in generating photorealistic visual content, they still struggle with the rigorous semantic alignment and logical reasoning required for scientific imagery. Inspired by Peirce's Semiotic Triad, we introduce Scientific Image Reasoning (SciIR), a comprehensive resource for training and evaluation of scientific image generation.
arXiv:2606. 29667v1 Announce Type: cross Abstract: The materials science literature encodes decades of experimental knowledge in figures, yet this visual record remains locked away and inaccessible to AI at scale.
arXiv:2511. 17731v2 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) prompting has proven remarkably effective for eliciting complex reasoning in large language models (LLMs).
The paper introduces TOPOGRAPHRAG-BENCH, a benchmark for evaluating multimodal Graph Retrieval-Augmented Generation (GraphRAG) systems on layout‑grounded evidence reasoning. It contains 2,024 questions derived from 201 long, visually rich documents, designed around three controlled topologies—single‑hop retrieval, bridge‑chain reasoning, and multi‑source synthesis—while employing counterfactual validation to ensure question integrity. Experiments show that multimodal GraphRAG outperforms text‑only and page‑level visual retrieval models, yet still struggles with incomplete visual‑textual alignment and multi‑unit composition, highlighting the need for GraphRAG systems that explicitly model document layouts and cross‑modal evidence alignment.
DocPilot is an agentic framework for long visual document understanding that orchestrates evidence seeking through page-level navigation and fine-grained localization, integrating visual inspection, code-based numerical reasoning, and cross-modal reasoning. It employs curriculum-based agentic training to enable small language models to learn complex document solving, progressing from single-ability learning to adaptive orchestration. Experiments on benchmarks such as MMLongBench-Doc, DocBench, FinRAGBench-V, and MADQA show that DocPilot achieves strong gains with frontier VLM backbones, and its 4B variant competes with much larger models, demonstrating effectiveness and generalization across diverse long-document reasoning tasks.
arXiv:2608. 03292v1 Announce Type: new Abstract: Long Document Visual Question Answering (LongDocVQA) requires Multimodal Large Language Models (MLLMs) to locate, integrate, and reason over heterogeneous document elements distributed across multiple pages.
MatMMExtract is an open‑source pipeline that disassembles compound scientific figures into individual sub‑panels and generates structured, grounded image‑text pairs using a large language model guided by a materials science taxonomy. Applied to 14,810 open‑access articles, it produced 391,606 panel‑level pairs with sub‑captions, a two‑level visualisation category (19 classes, 100+ subtypes), and scientific summaries. The project also introduces MaterialScope, a 2,811‑figure detection dataset, and demonstrates that Gemini 3.1 Flash Lite yields high‑quality annotations with low hallucination, while a dual‑encoder baseline outperforms zero‑shot CLIP on the resulting MatSciFig dataset.