The paper introduces SciGram, a large-scale dataset of 194K scientific diagrams paired with 1.4M visual instructions generated through a terminology‑grounded pipeline that extracts domain concepts, synthesizes facts, and retrieves relevant diagrams. Models fine‑tuned on SciGram show significant gains on diagram‑centric benchmarks such as TQA, ScienceQA, and AI2D, and when combined with existing models like LLaVA OneVision, set new state‑of‑the‑art performance. The authors release both the dataset and trained models to support further research in scientific diagram understanding.
By Raul Ortega, Jos\'e Manuel G\'omez-P\'erez
arXiv:2606. 28406v1 Announce Type: new Abstract: Text-to-image and multimodal generative models are increasingly used to produce scientific figures such as mechanism diagrams, experimental-design schematics, conceptual frameworks, and graphical abstracts.
By Davie Chen
arXiv:2606. 29667v1 Announce Type: cross Abstract: The materials science literature encodes decades of experimental knowledge in figures, yet this visual record remains locked away and inaccessible to AI at scale.
By Subham Ghosh, Shubham Tiwari, Mohammad Ibrahim, Abhishek Tewari
FigAct transforms static scientific figures into question‑conditioned visual presentations by acting directly on existing graphical elements. The framework generates short narrations, grounds each narration in visual evidence, and applies visual actions to guide viewer attention, mimicking a human presenter. A hierarchical search strategy reduces token usage by about 40×, and FigAct‑8B is trained with rewards for grounding accuracy, search efficiency, and rendering quality, evaluated on a human‑verified benchmark of real‑world scientific figures.
By Shishi Xiao, Zichao Wang, Alexa Siu, David H. Laidlaw, Jennifer Healey
The paper introduces Auto-Comp, a fully automated, concept-driven pipeline that generates photorealistic compositional benchmarks for vision‑language models. Auto‑Comp creates paired Minimal and Contextual samples for each concept, enabling isolation of core binding abilities from visio‑linguistic complexity. Evaluations across 25 models reveal consistent failures in attribute and relational binding, with context helping relational tasks but hindering attribute tasks due to visual clutter.
By Cristian Sbrolli, Toshihiko Yamasaki, Matteo Matteucci
arXiv:2610.14173v1 Announce Type: cross
Abstract: Digital libraries increasingly host scientific publications whose figures encode primary research findings, yet making this visual content searchable...
By Daniel J. Evans, J. P. Naiman, JooYoung Seo, J. Stephen Downie