arXiv AI

SketchVLM: Vision language models can annotate images to explain thoughts and guide users

arXiv Computation and Language
Sep 25

Multimodal Thinking with Renderable Programs

The paper introduces SVGLM, a framework that integrates scalable vector graphics (SVG) primitives into vision‑language models to enable image generation within reasoning tasks. By treating SVG both as image descriptions and text instructions, SVGLM offers a compact and interpretable method for connecting text and image reasoning. The authors provide a curated SVG‑based image editing dataset and demonstrate strong SVG generation and image‑aware reasoning performance on a mathematical benchmark.

By Sunli Chen, Ding Zhong, Ziqiao Ma, Jiaxin Liu, Zeyuan Yang, Hao Zhang, Lie Lu, Joyce Chai, Chuang Gan
arXiv Computer Vision
Oct 12

GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models

GraphVerse is a new benchmark for multimodal large language models that tests visual graph reasoning by combining perception, graph-based visual inputs, and multi-step reasoning. It introduces Graph-centric Image Editing strategies to actively challenge models while preserving semantics, and offers a VGR-Score metric that evaluates reasoning quality beyond simple answer accuracy. Experiments show current models have limitations in VGR and demonstrate the effectiveness of GraphVerse for broader multimodal reasoning tasks.

By Yuanfu Sun, Yuanhang Ren, Kang Li, Chuanhao Ji, Jiaxi Li, Jiajin Liu, Ninghao Liu, Qiaoyu Tan
arXiv AI
Jul 29

Visual prompt engineering for video models

arXiv:2607. 25537v1 Announce Type: cross Abstract: In the age of foundation models, a model is only as good as its prompt.

By Robert Geirhos, Yuxuan Li, Thadd\"aus Wiedemer, Neha Kalibhat, Zi Wang, Mani Malek, Oyvind Tafjord, Kevin Swersky, Been Kim, Priyank Jaini
arXiv AI
Sep 2

VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing

arXiv:2603.29852v2 Announce Type: replace-cross Abstract: We introduce VectorGym, a comprehensive benchmark suite for Scalable Vector Graphics (SVG) that spans generation from text and sketches, comp...

By Joan Rodriguez, Haotian Zhang, Abhay Puri, Haoran Dai, Tianyang Zhang, Meng Lin, Rishav Pramanik, Xiaoqing Xie, Marco Terral Rodriguez, Darsh Kaushik, Aly Shariff, Perouz Taslakian, Spandana Gella, Sai Rajeswar, David Vazquez, Christopher Pal, Marco Pedersoli
arXiv AI
Sep 10

A Progressive Training Strategy for Embodied Vision-Language Models to Mitigate Spatio-Temporal Hallucinations

The paper introduces a progressive training strategy for embodied vision‑language models aimed at reducing spatio‑temporal hallucinations. It first creates a Chain‑of‑Thought dataset that breaks complex reasoning into detailed spatiotemporal steps, then uses supervised pre‑training on this dataset followed by fine‑tuning with weakly‑labeled data. Experiments show the method improves backbone accuracy and narrows the forward‑backward performance gap from over 70% to 6.53%, indicating stronger dynamic reasoning and fewer temporal biases.

By Xiaoda Yang, Shuai Yang, Can Wang, Jingyang Xue, Menglan Tang, Checheng Yu, Xunzhe Zhou, Sashuai Zhou, Tao Jin, Lixin Yang, Xiangyu Yue, Zhou Zhao
arXiv Computer Vision
Oct 2

Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models

The paper introduces GTA‑VLA, an interactive Vision‑Language‑Action framework that lets users guide robot policies with explicit visual cues such as affordance points, boxes, and traces. Unlike traditional direct sense‑to‑act models, GTA‑VLA incorporates a spatial‑visual Chain‑of‑Thought that blends human guidance with internal task planning, and couples this reasoning module with a lightweight reactive action head for efficient execution. Experiments on the SimplerEnv WidowX benchmark show a state‑of‑the‑art 81.2 % success rate, and the framework significantly improves task success under out‑of‑domain visual shifts and spatial ambiguities, demonstrating the benefit of interactive reasoning for failure recovery in embodied control.

By Yiran Ling, Qing Lian, Jinghang Li, Qing Jiang, Tianming Zhang, Xiaoke Jiang, Chuanxiu Liu, Jie Liu, Lei Zhang