Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

8,506 stories · RSS feed

arXiv Computation and Language
Sep 1

Can MLLMs Critique Like Humans? Evaluating Open-Ended Aesthetic Reasoning in Multimodal Large Language Models

The study evaluates whether multimodal large language models (MLLMs) can produce open‑ended aesthetic critiques comparable to humans. Eight open‑weight MLLMs (7 B–397 B) and GPT‑5.5 were tested on 1,227 r/photocritique posts under various prompts, revealing that reference‑based similarity metrics often misrepresent model performance, while shorter critiques and image‑omission had limited impact. Human judges and annotators found the models’ critiques largely different from human ones, noting that models tend to be overly comprehensive and repetitive rather than selective and specific.

By Sajjad Ghiasvand, Maryam Amirizaniani, Haniyeh Ehsani Oskouie, Mahnoosh Alizadeh, Ramtin Pedarsani
arXiv AI
Sep 1

Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes

arXiv:2506.09557v2 Announce Type: replace-cross Abstract: While Multimodal Large Language Models (MLLMs) demonstrate impressive performance in benign scenarios, their cognitive reliability deteriorat...

By Zhaoyang Wei, Bowen Jiang, Xumeng Han, Jiashu Li, Xuehui Yu, Yuling Liu, Guorong Li, Zhenjun Han, Jianbin Jiao
arXiv Computer Vision
Sep 1

Chat-Edit-3D++: Interactive 3D and 4D Scene Editing via Large Language Models

The paper introduces Chat-Edit-3D++ (CE3D++), an interactive 3D and 4D scene editing system that uses a Hash-Atlas network to separate 2D editing from 3D reconstruction. CE3D++ employs a large language model to accept arbitrary textual input, interpret user intent, and autonomously invoke appropriate visual models, enabling multi‑round dialogue and diverse editing effects. The approach is extended to monocular 4D scenes by adding motion constraints and a trajectory dataset, allowing a smaller LLM to schedule up to 30 visual tools accurately.

By Shuangkang Fang, Yufeng Wang, Yi-Hsuan Tsai, Wenrui Ding, Yi Yang, Shuchang Zhou, Ming-Hsuan Yang
arXiv Computation and Language
Sep 1

UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models

UReason is a benchmark that evaluates how well unified multimodal models (UMMs) align textual reasoning with image generation. It contains 2,000 human‑curated instances across five reasoning‑intensive tasks—Code, Arithmetic, Spatial, Attribute, and Text—and compares direct generation, reasoning‑guided generation, and decontextualized generation. The study finds that while reasoning‑guided generation improves over direct generation, decontextualized generation consistently outperforms it, indicating that the visual semantics in textual reasoning are not reliably reflected in the generated images.

By Cheng Yang, Chufan Shi, Bo Shui, Yaokang Wu, Muzi Tao, Huijuan Wang, Ivan Yee Lee, Yong Liu, Xuezhe Ma, Taylor Berg-Kirkpatrick
arXiv Computation and Language
Sep 1

Mechanistic Diagnostics of Spatial Lexical Bias in Multimodal Large Language Model Spatial Reasoning

arXiv:2606.01914v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) remain unreliable on spatial multiple-choice questions, and their failures are often attributed to poorly...

By Chuang Ma, Qianying Liu, Tomoyuki Obuchi, Fei Cheng, Wang Yang, Sudong Cai, Shuyuan Zheng, Akiko Aizawa, Sadao Kurohashi
arXiv AI
Sep 1

MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents

arXiv:2508.13186v2 Announce Type: replace-cross Abstract: AI agents with advanced reasoning and tool-use capabilities have demonstrated impressive performance in web browsing for deep search. However...

By Shilong Li, Xingyuan Bu, Wenjie Wang, Jiaheng Liu, Jun Dong, Haoyang He, Hao Lu, Haozhe Zhang, Chenchen Jing, Zhen Li, Chuanhao Li, Jiayi Tian, Chenchen Zhang, Tianhao Peng, Yancheng He, Jihao Gu, Hui Huang, Donghao Zhou, Yuanxing Zhang, Jian Yang, Ge Zhang, Wenhao Huang, Zhaoxiang Zhang, Qiangpeng Yang, Shilei Wen
arXiv AI
Sep 1

LeMat-Synth: a multi-modal toolbox to curate broad synthesis procedure databases from scientific literature

arXiv:2510.26824v2 Announce Type: replace-cross Abstract: Wide access to advanced experimental methods in materials science has given rise to an abundance of procedural knowledge, which is scattered...

By Magdalena Lederbauer, Siddharth Betala, Valerie Gentzke, Anamaria Leonescu, Amine Sehaba, Faris Flaifil, Ayush Jain, Alfonso Amayuelas, Nikhil Yelamarthy, Xiyao Li, Gr\'egoire Germain, Stefano Ribes, Stefan P. Schmid, Alexandre Nozadze, Anna Kelmanson, Sudheesh Kumar Ethirajan, Mohd Zaki, Elton Pan, Georgia Channing, Connor W. Coley, Philippe Schwaller, Roc\'io Mercado, Alexandre Duval, Mathilde L. D. Franckel, Samuel P. Gleason
arXiv Computer Vision
Sep 1

Can Retrieval Heads See Images? Multimodal Retrieval Heads in Long-Context Vision-Language Models

arXiv:2605.27243v3 Announce Type: replace Abstract: Large vision-language models increasingly rely on long-context modeling to reason over documents, hour-level videos, and long-horizon agent traject...

By Aaron Branson Cigres Li, Zhaowei Wang, Yu Zhao, Yiming Du, Haobo Li, Xiyu Ren, Ginny Wong, Simon See, Lishu Luo, Haodong Duan, Pasquale Minervini, Yangqiu Song
arXiv AI
Sep 1

VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing

arXiv:2605.30117v2 Announce Type: replace Abstract: Understanding how Vision-Language-Action (VLA) models transform multimodal knowledge into embodied control remains an open challenge. We present VL...

By Haoyuan Shi, Xiancong Ren, Yingji Zhang, Qinfan Zhang, Jiayu Hu, Haozhe Shan, Han Dong, Jinpeng Lu, Yinda Chen, Yi Zhang, Yong Dai, Xiaozhu Ju