The paper surveys Multimodal Code Intelligence, focusing on tasks where code is generated, edited, refined, or reasoned about under visually grounded inputs such as screenshots, charts, and videos. It categorizes the field by the role of code—rendered artifact, editable structure, intermediate reasoning trace, or executable tool interface—and organizes benchmarks into four domains: Graphical User Interface, Scientific Visualization, Structured Graphics, and Frontier Tasks and Frameworks. The authors argue that reliable evaluation must include evidence of semantics and interaction beyond visual fidelity, and propose four verification-centered research directions to advance the field toward evidence-grounded executable systems.
By Xuanle Zhao, Qiushi Sun, Jingyu Xiao, Xuexin Liu, Haoyue Yang, Qiaosheng Chen, Xianzhen Luo, Jing Huang, Yufeng Zhong, Lei Chen, Shuai Fu, Zhenlin Wei, Jinhe Bi, Lei Jiang, Haibo Qiu, Siqi Yang, Peng Shi, Jian Hu, Zhixiong Zeng
The paper introduces KVoiceBench, KOpenAudioBench, and KMMAU—three Korean speech benchmarks created through agent-driven frameworks that adapt existing SpokenQA and ASR resources into Korean SpokenQA and audio understanding tasks. These benchmarks total 12,345 samples and are publicly released to evaluate SpeechLMs beyond English. The authors benchmark eight recent SpeechLMs, revealing significant English‑Korean performance gaps and divergent rankings between SpokenQA and audio understanding, highlighting multilingual weaknesses not apparent in English-only tests.
By Haechan Kim, Seungjun Chung, Inkyu Park, Jihoo Lee, Jonghyun Lee
The study shows that jailbreak robustness in language models is highly sensitive to operational-state changes. Even minor alterations to system prompts, not intended to affect safety, can dramatically shift attack success rates across seven aligned models and three jailbreak methods. The authors link these variations to changes in hidden representations along a refusal-related axis, which can predict jailbreak outcomes.
By Yuna Park, Hwang Youn Kim, Yujin Kim, Won Woo Ro, Suhyun Kim, Jae-In Hwang
SpanCalib-VLM is a hybrid system for detecting hallucinated text spans in Vision‑Language Models. It combines a multimodal sequence tagger (XLM‑RoBERTa‑Large + SigLIP) with a fine‑tuned generative VLM (Qwen3.5‑4B‑SHROOM‑SFT) and uses a Union‑Calibrated Fusion strategy to re‑score candidate spans. On the SHROOM‑Visions English evaluation split, the ensemble achieves a Pearson calibration correlation of 0.41, an overall IoU of 0.39, a clean‑response IoU of 0.91, and a detection accuracy of 70.7%.
By Amanuel Gizachew Abebe, Yasmin Moslem
LOC I (Locator‑Critic) is a training‑free framework that separates visual search from evidence verification in Vision‑Language Models. It uses a Locator agent to propose candidate visual evidence and a Critic agent to assess its relevance, engaging in an iterative refinement loop that progressively improves the evidence until it is sufficient to answer a question. The approach yields state‑of‑the‑art results on several complex visual benchmarks, boosting accuracy for both open‑weight models like Qwen3‑VL and proprietary models such as Gemini 2.5 Pro.
By Walid Bousselham, Mathilde Caron, Arsha Nagrani, Cordelia Schmid
MMDS-Bench is a new diagnostic benchmark for multimodal dynamic stance classification in social media parent‑reply interactions. It contains 3,482 multimodal instances annotated with a seven‑label stance taxonomy, plus an 800‑instance subset that demands structured reasoning over parent and reply understanding and stance‑relation inference. The benchmark also tags each instance with five challenge factors—multimodal fusion, parent framing, non‑literal expression, interaction reasoning, and label‑boundary ambiguity—and evaluates 12 multimodal large language models using a reference‑grounded LLM‑judge protocol, revealing that current models still struggle with relational inference beyond separate parent and reply comprehension.
By Yuzhe Ding, Kang He, Li Zheng, Shengwu Zheng, Teng Shi, Fei Li, Chong Teng, Donghong Ji
arXiv:2608.28981v1 Announce Type: new
Abstract: As air traffic volumes in the National Airspace System continue to expand, in particular in the low altitude airspaces, the need for scalable decision...
By Louis Brusset, Mathurin Petit, Jordan Kam, Alexandre Bayen
arXiv:2608.30498v1 Announce Type: new
Abstract: Multimodal Large Language Models (MLLMs) have shown remarkable success in STEM domains, where progress is often driven by vertical, step-by-step deduct...
By Qi Li, Zhaojie Kang, Yingjie He, Zheng Lin, Hao Zhang, Guangxin Wu, Yan Gong, Rong Fu, Jianyuan Ni
Co-Annotator is a clinical AI system that distills expert gaze and dictation into two guidance components: a gaze‑aligned Vision Transformer that highlights fixation‑aligned areas of interest (AOIs) and an ontology‑bounded vision‑language model that pre‑fills editable biomarker summaries for retinal OCT. In controlled studies, each modality independently improved diagnostic accuracy and biomarker generation, and when combined across two academic institutions, the system increased correct diagnoses per minute by 40% and reduced comment editing time by 67% without compromising accuracy.
By Ziheng "Leo" Li, Benjamin Freeman, Akshay Raman, Kavin Aravindhan Rajkumar, Xinxin Fang, Rishabh Srivastava, Steven Feiner, Kaveri A. Thakoor
arXiv:2608.30726v1 Announce Type: new
Abstract: Multimodal Sentiment Analysis (MSA) is a fundamental component of affective computing that aims to decipher complex emotional states by integrating ver...
By Xiaode Chen, Jiakang Yu, Hongtao Deng, Huina Qu, Xun Zhu, Yinxia Lou
arXiv:2608.31022v1 Announce Type: new
Abstract: AI agents in partially observable environments need to coordinate active sensing with working memory to maintain an evolving perceptual state. However,...
By Vernon Toh, Navonil Majumder, Zhengyuan Liu, Nancy F. Chen, Soujanya Poria
arXiv:2608.30475v1 Announce Type: cross
Abstract: We present an overview of the ImageEval 2026 shared task on culturally grounded Arabic multimodal evaluation. It includes two tasks: (i) AynVQA, cove...
By Samir Abdaljalil, Hunzalah Hassan Bhatti, Ahlam Bashiti, Farina Amir, Md Arid Hasan, Basel Mousi, Nadir Durrani, Fahim Dalvi, Zien Sheikh Ali, Erchin Serpedin, Hasan Kurban, Mustafa Jarrar, Shammur Absar Chowdhury, Firoj Alam
arXiv:2605.07593v2 Announce Type: replace
Abstract: Real-world audio-visual understanding requires chaining evidence that is sparse, temporally dispersed, and split across the visual and auditory str...
By Hengyi Feng, Hao Liang, Mingrui Chen, Bohan Zeng, Meiyi Qiang, Zhengyang Zhao, Zimo Meng, Zeang Sheng, Wentao Zhang
arXiv:2604.22875v3 Announce Type: replace-cross
Abstract: When answering questions about images, humans naturally point, label, and draw to explain their reasoning. In contrast, modern vision-languag...
By Brandon Collins, Logan Bolton, Hung Huy Nguyen, Mohammad Reza Taesiri, Trung Bui, Anh Totti Nguyen
The paper introduces Image Bundle Composition (IBC), a new paradigm that moves beyond point-wise image matching to dynamically assemble cohesive image bundles from large, unstructured photo collections. It presents IBCBench, a benchmark with over 109,000 images and 667 verified queries, and proposes BundleWeaver, an agentic framework that uses a Large Language Model for relational role discovery and a Vision‑Language Model for bundle verification. Experiments show that BundleWeaver outperforms existing embedding and decompose‑and‑rerank methods, underscoring the importance of relational composition over atomic scoring.
By Rong Shan, Tianyi Xu, Congmin Zheng, Wenteng Chen, Jiachen Zhu, Junjie Wu, Teng Wang, Weiwen Liu, Changwang Zhang, Weinan Zhang, Jun Wang, Jianghao Lin
arXiv:2603.26737v2 Announce Type: replace-cross
Abstract: Current multimodal LLMs encode images as static visual prefixes and rely on text-based reasoning, lacking goal-driven and adaptive visual acc...
By Guangfu Guo, Xiaoqian Lu, Yue Feng, Mingming Sun
arXiv:2509.17930v3 Announce Type: replace-cross
Abstract: Multilingual translation suffers from computational redundancy, especially when translating into multiple languages simultaneously. In additi...
By Yiwen Guan, Jacob Whitehill
arXiv:2608.30260v1 Announce Type: cross
Abstract: While it is well-established that prosody carries crucial cues for syntactic structure, the degree and nature of correspondence between these two dom...
By Junghyun Min, Alex Warstadt, Tamar I. Regev, Tiago Pimentel, Ethan Gotlieb Wilcox
arXiv:2503.00610v2 Announce Type: replace-cross
Abstract: Understanding how people perceive urban environments is essential for inclusive planning, yet conventional surveys are costly and difficult t...
By Ciro Beneduce, Bruno Lepri, Massimiliano Luca
arXiv:2510.06113v2 Announce Type: replace
Abstract: Survival analysis plays a vital role in making clinical decisions. However, the models currently in use are often difficult to interpret, which red...
By Shuo Jiang, Zhuwen Chen, Liaoman Xu, Yanming Zhu, Changmiao Wang, Jiong Zhang, Feiwei Qin, Yifei Chen, Zhu Zhu