The paper introduces SciGram, a large-scale dataset of 194K scientific diagrams paired with 1.4M visual instructions generated through a terminology‑grounded pipeline that extracts domain concepts, synthesizes facts, and retrieves relevant diagrams. Models fine‑tuned on SciGram show significant gains on diagram‑centric benchmarks such as TQA, ScienceQA, and AI2D, and when combined with existing models like LLaVA OneVision, set new state‑of‑the‑art performance. The authors release both the dataset and trained models to support further research in scientific diagram understanding.
By Raul Ortega, Jos\'e Manuel G\'omez-P\'erez
arXiv:2606. 28406v1 Announce Type: new Abstract: Text-to-image and multimodal generative models are increasingly used to produce scientific figures such as mechanism diagrams, experimental-design schematics, conceptual frameworks, and graphical abstracts.
By Davie Chen
arXiv:2606. 29667v1 Announce Type: cross Abstract: The materials science literature encodes decades of experimental knowledge in figures, yet this visual record remains locked away and inaccessible to AI at scale.
By Subham Ghosh, Shubham Tiwari, Mohammad Ibrahim, Abhishek Tewari
FigAct transforms static scientific figures into question‑conditioned visual presentations by acting directly on existing graphical elements. The framework generates short narrations, grounds each narration in visual evidence, and applies visual actions to guide viewer attention, mimicking a human presenter. A hierarchical search strategy reduces token usage by about 40×, and FigAct‑8B is trained with rewards for grounding accuracy, search efficiency, and rendering quality, evaluated on a human‑verified benchmark of real‑world scientific figures.
By Shishi Xiao, Zichao Wang, Alexa Siu, David H. Laidlaw, Jennifer Healey
The paper introduces Auto-Comp, a fully automated, concept-driven pipeline that generates photorealistic compositional benchmarks for vision‑language models. Auto‑Comp creates paired Minimal and Contextual samples for each concept, enabling isolation of core binding abilities from visio‑linguistic complexity. Evaluations across 25 models reveal consistent failures in attribute and relational binding, with context helping relational tasks but hindering attribute tasks due to visual clutter.
By Cristian Sbrolli, Toshihiko Yamasaki, Matteo Matteucci
arXiv:2610.14173v1 Announce Type: cross
Abstract: Digital libraries increasingly host scientific publications whose figures encode primary research findings, yet making this visual content searchable...
By Daniel J. Evans, J. P. Naiman, JooYoung Seo, J. Stephen Downie
SciDiagBench is a new benchmark designed to evaluate scientific diagram generation in a way that mirrors human peer‑review practices. It provides a dataset of 165 diagrams from 10 domains, systematically classified into 11 distinct types such as graphical abstracts and technical roadmaps. The benchmark also introduces a preference‑guided calibration method that incorporates human preferences into type‑specific evaluation criteria, aiming to improve the reliability of automated assessments.
By Hao Wang, Zixuan Li, Long Bai, Xiaolong Jin, Jiafeng Guo, Tat-Seng Chua, Tian Tian
arXiv:2610.20083v1 Announce Type: new
Abstract: Visual world knowledge is a core capability of multimodal large language models (MLLMs), underpinning knowledge-intensive visual question answering and...
By Chen Chen, Zheming Liang, Jiaqi Wang
arXiv:2605.26380v2 Announce Type: replace-cross
Abstract: Frontier multimodal large language models (MLLMs) have been reported to achieve over 90\% accuracy on fine-grained perception benchmarks. How...
By Jingru Chen, Yiming Liu, Mingtao Chen, Sijie Chen, Richeng Xuan, Liang Yang, Zhichao Hu, Fanyang Lu
OneSearch‑VL is a unified multimodal deep research agent that handles single‑image, multi‑image, and video tasks by leveraging a Visually Grounded Evidence Graph (VGEG) to encode dependencies among visual anchors, entity relations, facts, and answer‑producing operations. The VGEG‑based data engine builds and verifies questions, filters expert trajectories, and supports supervised fine‑tuning (SFT) and reinforcement learning (RL) with the Evidence‑aware Visual‑Grounded Rubric reward (EVGR). Experiments demonstrate that OneSearch‑VL‑8B outperforms Qwen3‑VL‑8B with tool access by 20.2 and 17.6 percentage points on new multimodal benchmarks, while also achieving significant gains across seven image benchmarks and VideoDR.
By Hongyu Li, Manyuan Zhang, Kaituo Feng, Shu Chen, Dian Zheng, Hao Li, Hao Yu, Zhangquan Chen, Zoey Guo, Ray Zhang, Shaofei Huang, Tianrui Hui, Linjiang Huang, Si Liu
MatMMExtract is an open‑source pipeline that disassembles compound scientific figures into individual sub‑panels and generates structured, grounded image‑text pairs using a large language model guided by a materials science taxonomy. Applied to 14,810 open‑access articles, it produced 391,606 panel‑level pairs with sub‑captions, a two‑level visualisation category (19 classes, 100+ subtypes), and scientific summaries. The project also introduces MaterialScope, a 2,811‑figure detection dataset, and demonstrates that Gemini 3.1 Flash Lite yields high‑quality annotations with low hallucination, while a dual‑encoder baseline outperforms zero‑shot CLIP on the resulting MatSciFig dataset.
By Subham Ghosh, Shubham Tiwari, Mohammad Ibrahim, Abhishek Tewari
arXiv:2607. 09068v1 Announce Type: cross Abstract: Recent advancements in LVLMs necessitate robust benchmarks for complex, visually grounded reasoning.
By Yang Chen, Yunwen Li, Yufan Shen, Minghao Liu, Tianyu Zheng, Bin Fu, Qunshu Lin, Zhi Yu, Botian Shi