arXiv Machine Learning

Symbolic Graphics Programming with Large Language Models

arXiv:2509. 05208v2 Announce Type: replace-cross Abstract: Large language models (LLMs) excel at program synthesis, yet their ability to produce symbolic graphics programs (SGPs) that render into precise visual content remains underexplored.

arXiv AI

Jun 3

TurtleAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics

arXiv:2606. 03626v1 Announce Type: cross Abstract: Vision-language models (VLMs) have been explored for visual programming, where they generate code to solve visual tasks.

By Chao Wen, Jacqueline Staub, Adish Singla

llms fine-tuning multimodal benchmarks safety

arXiv AI

1d ago

VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models

arXiv:2603. 24575v2 Announce Type: replace-cross Abstract: Scalable Vector Graphics (SVG) are essential for technical illustration and digital design, offering resolution independence and semantic editability.

By Qijia He, Xunmei Liu, Hammaad Memon, Ziang Li, Zixian Ma, Jaemin Cho, Zhongzheng Ren, Daniel S Weld, Ranjay Krishna

llms rag reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Jun 6

Image Generators are Generalist Vision Learners

arXiv:2604. 20329v3 Announce Type: replace-cross Abstract: Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language understanding and reasoning from generative pretraining.

By Valentin Gabeur, Shangbang Long, Songyou Peng, Paul Voigtlaender, Shuyang Sun, Yanan Bao, Karen Truong, Zhicheng Wang, Wenlei Zhou, Jonathan T. Barron, Kyle Genova, Nithish Kannen, Sherry Ben, Yandong Li, Mandy Guo, Suhas Yogin, Yiming Gu, Huizhong Chen, Oliver Wang, Saining Xie, Howard Zhou, Kaiming He, Thomas Funkhouser, Jean-Baptiste Alayrac, Radu Soricut

llms diffusion computer-vision benchmarks

arXiv AI

Jul 7

SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

arXiv:2511. 07403v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language tasks, but continue to struggle with spatial reasoning.

By Hunar Batra, Haoqin Tu, Hardy Chen, Yuanze Lin, Cihang Xie, Ronald Clark

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Jun 3

Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Models

arXiv:2605. 18160v2 Announce Type: replace-cross Abstract: In recent years, multimodal large language models (MLLMs) have achieved remarkable progress, primarily attributed to effective paradigms for integrating visual and textual information.

By Xinpeng Dong, Min Zhang, Kairong Han, Xu Tan, Fei Wu, Kun Kuang

llms computer-vision multimodal benchmarks safety

Hugging Face Trending Papers

Jul 7

Vision as Unified Multimodal Generation

We formulate computer vision as unified multimodal generation, where heterogeneous visual tasks are expressed in the native text and image generation spaces of a unified multimodal model, without task-specific architectures. Under this formulation, SenseNova-Vision uses natural-language instructions and optional visual prompts to specify tasks, target regions or views, and decoding conventions, and generates responses as text for symbolic outputs, images for dense spatial predictions, or mixed text-and-image outputs for compositional tasks.

diffusion computer-vision multimodal

arXiv AI

Jul 7

MentalThink: Shaping Thoughts in Mental SVG World

arXiv:2607. 03530v1 Announce Type: new Abstract: We introduce MentalThink, a visual-symbolic reasoning paradigm that equips Multimodal LLMs (MLLMs) with an executable mechanism for "mental" visualization.

By Kangheng Lin, Jisheng Yin, Dingming Li, En Yu, Yana Wei, Han Zhou, Liang Zhao, Hongyu Zhou, Hongbo Peng, Jianjian Sun, Zheng Ge, Xiangyu Zhang, Daxin Jiang, Jingyu Wang

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Jul 21

Think, Plan, Paint: Layout-Aware Reasoning for Controllable Image Generation in Unified Models

arXiv:2607. 16409v1 Announce Type: cross Abstract: Unified Multimodal Large Language Models (MLLMs) offer a promising paradigm for unifying visual understanding and generation, yet they still struggle to follow complex spatial instructions and logical constraints in controllable image generation.

By Junhao Liu, Jian-Wei Zhang, Tao Huang, Miles Yang, Zhao Zhong, Liefeng Bo

llms diffusion multimodal benchmarks safety

arXiv AI

Aug 7

CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation

arXiv:2603. 08652v2 Announce Type: replace Abstract: Recent advancements in Unified Multimodal Models (UMMs) have significantly advanced text-to-image (T2I) generation, particularly through the integration of Chain-of-Thought (CoT) reasoning.

By Haodong Li, Chunmei Qing, Huanyu Zhang, Dongzhi Jiang, Yihang Zou, Hongbo Peng, Dingming Li, Yuhong Dai, ZePeng Lin, Juanxi Tian, Yi Zhou, Siqi Dai, Jingwei Wu, Pheng-Ann Heng

diffusion multimodal

arXiv Machine Learning

5d ago

From Visual Widgets to UI Code: Efficient Tool-Grounded Generation

arXiv:2608. 12611v1 Announce Type: cross Abstract: Existing screenshot-to-code systems face a trade-off between flexibility and controllability.

By Houston H. Zhang, Tao Zhang, Li Gu, Linfeng Ye, Yuanhao Yu, Xinxin Zuo, Yang Wang, Zhixiang Chi

llms agents fine-tuning multimodal

arXiv AI

Jul 15

CrochetBench: Can Vision-Language Models Move from Describing to Doing in Crochet Domain?

arXiv:2511. 09483v3 Announce Type: replace Abstract: While multimodal large language models can describe visual content, their ability to generate executable procedures remains underexplored.

By Peiyu Li, Xiaobao Huang, Ting Hua, Nitesh V. Chawla

llms multimodal benchmarks

arXiv AI

Jun 11

Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training

arXiv:2606. 11854v1 Announce Type: cross Abstract: There are two main Parameter-Efficient Fine-Tuning (PEFT) techniques for Large Language Models (LLMs).

By Michal Chudoba, Sergey Alyaev, Petra Galuscakova, Tomasz Wiktorski

llms fine-tuning efficiency multimodal benchmarks

Related stories