arXiv Computation and Language By Hao Wang, Zixuan Li, Long Bai, Xiaolong Jin, Jiafeng Guo, Tat-Seng Chua, Tian Tian

SciDiagBench: A Type-Specific Benchmark for Human-Aligned Evaluation of Scientific Diagrams

Read the original on arXiv Computation and Language →

SciDiagBench is a new benchmark designed to evaluate scientific diagram generation in a way that mirrors human peer‑review practices. It provides a dataset of 165 diagrams from 10 domains, systematically classified into 11 distinct types such as graphical abstracts and technical roadmaps. The benchmark also introduces a preference‑guided calibration method that incorporates human preferences into type‑specific evaluation criteria, aiming to improve the reliability of automated assessments.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI
Jun 30

SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents

arXiv:2603. 29139v2 Announce Type: replace Abstract: Recent advances in large language models (LLMs) have enabled agentic systems to translate natural-language intent into executable scientific visualization (SciVis) tasks.

By Kuangshi Ai, Haichao Miao, Kaiyuan Tang, Nathaniel Gorski, Jianxin Sun, Guoxi Liu, Helgi I. Ingolfsson, David Lenz, Hanqi Guo, Hongfeng Yu, Teja Leburu, Michael Molash, Bei Wang, Tom Peterka, Chaoli Wang, Shusen Liu
arXiv Computation and Language
Oct 21

SciFigBench: A Benchmark for Scientific Method Figure Understanding

SciFigBench is a new benchmark for evaluating Vision Language Models on scientific method figure understanding. It focuses on three fine‑grained capabilities—dense content perception, visual relation comprehension, and information‑flow tracing—using 4,352 curated figures and 40,670 QA pairs. The authors also introduce SciFigCue, a training‑free visual prompting module that improves performance on several tasks without updating model parameters.

By Lijia Wu, Zheng Li, Yuxuan Liu, Jingwen Zhang, Penghai Zhao, Xiang Li
arXiv AI
Sep 2

From Terminology to Diagrams: Visual-Instruction Generation for Scientific Diagram Understanding

The paper introduces SciGram, a large-scale dataset of 194K scientific diagrams paired with 1.4M visual instructions generated through a terminology‑grounded pipeline that extracts domain concepts, synthesizes facts, and retrieves relevant diagrams. Models fine‑tuned on SciGram show significant gains on diagram‑centric benchmarks such as TQA, ScienceQA, and AI2D, and when combined with existing models like LLaVA OneVision, set new state‑of‑the‑art performance. The authors release both the dataset and trained models to support further research in scientific diagram understanding.

By Raul Ortega, Jos\'e Manuel G\'omez-P\'erez