CS-CLIP is a vision‑language model that improves compositional reasoning by using scene graphs to identify compositional elements and create structured negative examples through selective masking. The approach retains only the most contradictory negatives, encouraging the model to depend on compositional structure instead of surface cues. CS-CLIP achieves state‑of‑the‑art performance on compositional reasoning benchmarks while maintaining strong cross‑modal retrieval and downstream visual reasoning capabilities with fewer training samples.
By SeongJun Jeong, Minjoon Jung, Woo Suk Choi, Youwon Jang, Byoung-Tak Zhang
The paper introduces Emo-DVS, a large-scale, multimodal dataset combining event camera, audio, and text data for emotion recognition, designed to mitigate privacy concerns associated with RGB cameras. It proposes the Information‑Guided Gated Fusion (IGF) framework, which pre‑trains an event encoder on the dataset’s FAU subset, adaptively gates modalities to reduce noise, and aligns cross‑modal representations via mutual information maximization. Experiments show that IGF outperforms existing methods on this challenging tri‑modal benchmark.
By Jiaqi Chen, Qinfu Xu, Hao Zhuang, Liyuan Pan
The paper introduces the Geo-Context Guided Visual Transformer, a model that augments remote sensing image analysis with geospatial embeddings and an asymmetric attention module. By converting heterogeneous geospatial variables into patch-aligned representations and assigning geospatial roles to attention heads, the approach improves disease prevalence prediction over existing vision-language and graph-based baselines. Ablation and visualization studies demonstrate its effectiveness and interpretability for health-related remote sensing tasks, especially when comprehensive geospatial data are scarce.
By Yu Li, Guilherme N. DeSouza, Praveen Rao, Chi-Ren Shyu
The paper introduces a multi‑modal late‑fusion perception pipeline for object detection and tracking in autonomous racing. It combines independent detections from cameras, LiDARs, and RADARs to produce timely and robust state estimates of surrounding vehicles. The tracking framework compensates for detection delays and incorporates vehicle dynamics and track layout knowledge, and its effectiveness is confirmed through real‑world experiments in diverse critical scenarios.
By Davide Malvezzi, Michele Pestarino, Vittoria Cavicchioli, Valentina La Gamba, Silvia Severi, Fabio Bagni, Luca Bartoli, Massimiliano Bosi, Francesco Gatti, Micaela Verucchi, Ayoub Raji, Marko Bertogna
GoDeep is an annotation‑free method for open‑vocabulary 3D scene understanding that uses a vision‑language model solely as a translator to generate structured, entity‑level descriptions of each image. These descriptions are projected and aggregated in a language‑only embedding space, eliminating the need for a 3D training corpus or domain‑specific encoder. The approach achieves competitive performance on ScanNet++ and a cultural heritage benchmark, accurately localizes out‑of‑vocabulary objects, and offers explainable, point‑level predictions.
By Thodoris Betsas, Anastasios Doulamis, Andreas Georgopoulos
SEA-SpeechBench is a large‑scale multitask benchmark for speech understanding in 11 Southeast Asian languages, comprising 97,194 samples across 99 evaluation sets and 597 hours of curated audio. It covers nine tasks in three categories—speech processing, paralinguistic analysis, and a novel temporal understanding dimension—using multilingual prompting in both native SEA languages and English. Evaluation of current models shows significant performance gaps, especially in temporal understanding, emotion recognition, and speech translation, with low‑resource languages lagging behind English by up to 41 percentage points.
By Jingyi Liao, Wenyu Zhang, Zhuohan Liu, Yingxu He, Geyu Lin, Xunlong Zou, Shuo Sun, Syed Ali Redha Alsagoff, Ai Ti Aw
The paper "Inferring the Unspoken: Aligning Embodied Agents with Implicit Preferences" addresses the challenge of natural-language instructions that omit details needed for embodied action. It introduces the Preference-based Planning (PbP) benchmark, comprising 5,000 evaluation groups and 290 preferences across three levels, to systematically evaluate agents’ ability to infer latent user preferences from a few demonstrations. The authors propose the two-stage Inferring the Unspoken (InTU) framework, which first verbalizes inferred preferences from multimodal demonstrations and then generates action plans conditioned on that explicit representation, showing that explicit verbalization improves alignment and robustness compared to direct end-to-end planning.
By Manjie Xu, Xinyi Yang, Wei Liang, Chi Zhang, Yixin Zhu
The paper introduces a progressive training strategy for embodied vision‑language models aimed at reducing spatio‑temporal hallucinations. It first creates a Chain‑of‑Thought dataset that breaks complex reasoning into detailed spatiotemporal steps, then uses supervised pre‑training on this dataset followed by fine‑tuning with weakly‑labeled data. Experiments show the method improves backbone accuracy and narrows the forward‑backward performance gap from over 70% to 6.53%, indicating stronger dynamic reasoning and fewer temporal biases.
By Xiaoda Yang, Shuai Yang, Can Wang, Jingyang Xue, Menglan Tang, Checheng Yu, Xunzhe Zhou, Sashuai Zhou, Tao Jin, Lixin Yang, Xiangyu Yue, Zhou Zhao
The paper introduces GlobeReady, a clinician-friendly platform that leverages the RetiGlobe foundation model for ophthalmic image diagnostics without requiring fine-tuning. RetiGlobe was pretrained in two stages: first with self-supervised learning on 38 million synthetic images, then with contrastive learning on 475,845 real image‑text pairs from diverse ethnicities, devices, and regions. GlobeReady was evaluated on 488,448 images from multiple international centers and tested prospectively with 31 ophthalmologists, also exploring domain generalisability, uncertainty quantification, OOD detection, and feature-based case retrieval.
By Meng Wang, Tian Lin, Qingshan Hou, Aidi Lin, Lianyu Wang, Jingcheng Wang, Qingsheng Peng, Truong X. Nguyen, Zhi Da Soh, Xiayin Zhang, Jingyan Yang, Danqi Fang, Ke Zou, Ting Xu, Can Can Xue, Ten Cheer Quek, Qinkai Yu, Minxin Liu, Hui Zhou, Zixuan Xiao, Guiqin He, Huiyu Liang, Tingkun Shi, Man Chen, Zhuangling Lin, Linna Liu, Yuanyuan Peng, Li Jia Chen, Chi Ming Chan, Xiaohong Li, Junren He, Zhirong Xu, Tingbing Fang, Yanli Wang, Qingzhi Wang, Wenyi Hu, Yujie Wang, Li Li, Jiaying Ye, Tonghui Ye, Liang Lyu, Yongjian Lu, Ruoshi Cai, Yiwen Tang, Qiuming Hu, Junhong Chen, Zhenhua Zhang, Cheng Chen, Yitian Zhao, Dianbo Liu, Jianhua Wu, Xinjian Chen, Changqing Zhang, Xiaojun Wu, Triet Thanh Nguyen, Yanda Meng, Yalin Zheng, Daoqiang Zhang, Xiaochun Cao, Yih Chung Tham, Ye Zhang, Ying Han, Alvin L Young, Mary Ho, Carmen K M Chan, Clement C Tham, Zhuoting Zhu, Carol Y. Cheung, Tien Yin Wong, Huazhu Fu, Haoyu Chen, Ching-Yu Cheng
The paper describes a system for the SHROOM-Visions 2026 shared task on character-level VLM hallucination detection. It combines a small 4‑B‑parameter VLM fine‑tuned as a per‑token classifier that uses a two‑token feature from its hidden states with a large ~400‑B zero‑shot VLM judge at prediction time, both leveraging OCR of visible in‑image text. Using synthetic hallucination data from the large model for ensemble diversity and validation‑based selection of feature layer, training data, and OCR grounding, the entry achieved competitive results across multiple languages.
By Eli Schwartz
CUSP (Collective Uncertainty through Semantic Opinion Pooling) is a training‑free framework that aggregates responses from multiple vision‑language models into a shared semantic space, producing a pooled opinion and two system‑level uncertainty signals: collective uncertainty (dispersion) and Jensen‑Shannon divergence (model conflict). It decomposes collective entropy into the mean of individual semantic entropies plus JSD, enabling reliable uncertainty estimation without token logits or calibration labels. In static ensembles, collective uncertainty outperforms baseline methods for error detection and abstention, while JSD excels in commercial settings, and the pooled prediction consistently improves accuracy over individual models.
"whyItMatters":"CUSP provides a practical, model‑agnostic way to quantify system‑level reliability and improve decision‑making in multimodal reasoning tasks."
By Chung-En Johnny Yu, David Garcia, Brian Jalaian, Nathaniel D. Bastian
The paper presents a layer‑wise gating mechanism for prompt truncation in a multimodal chest X‑ray classifier based on Mixture of Prompt Experts (MoPE). The controller predicts a retention ratio per sample, averages these ratios per mini‑batch, and truncates the static and mixed visual prompts to an integer length, also scaling retained prompts by the individual ratios. In a pilot study, the gated model achieved a best validation accuracy of 0.8996 versus 0.8969 for the fixed‑length baseline, with final values of 0.8963 and 0.8802, while the gate statistics showed a retained length of one token out of a configured maximum of six, reducing the visual sequence from 210 to 200 tokens.
"whyItMatters":"The study demonstrates that prompt shortening via a gating strategy can slightly improve validation accuracy in a multimodal chest X‑ray classification task, though it does not yet confirm runtime acceleration or clinical utility."
By Jingtao Lei, Hongji Li, Dexiang Shu
The paper proposes a lightweight federated multiple‑instance learning (MIL) framework that trains only a compact MIL scorer across distributed clients while using a frozen vision‑language model (VLM) to verify high‑scoring video segments post‑hoc. Two VLM feedback interfaces are explored: a parsed text‑generation interface and a logit‑based interface that derives a continuous anomaly score from next‑token Yes/No probabilities. Experiments on UCF‑Crime with InternVL3.5‑2B and Qwen3‑VL‑2B‑Instruct show that the logit interface consistently improves frame‑level AUC and AP over the MIL baseline without requiring temporal post‑processing, whereas the text‑generation interface is more sensitive to prompts, parsers, and model choice.
By S\'ebastien Thuau, Amira Gran, Siba Haidar, Rachid Chelouah
The paper introduces RouteCert, a method for ensuring risk control in multimodal systems that acquire inputs adaptively. It shows that conditional calibration can remain valid even when the acquisition policy determines the calibration group, and provides two finite‑sample constructions: threshold‑free routing with terminal‑pattern calibration and simultaneous validation of policy‑pattern pairs. Experiments on a clinical ECG task and masked multimodal benchmarks demonstrate that RouteCert achieves low disagreement rates and competitive answered fractions while validating each acquisition stage separately.
By Melika Baghi
The paper introduces Mind2Web-Injection, a benchmark of 9,954 instruction‑screenshot pairs designed to evaluate whether vision‑language models (VLMs) use visual evidence when judging conflicts between on‑screen text and user instructions. Across six VLMs, models with similar overall precision can differ dramatically in Evidence‑Aligned Detection (EAD), revealing that verdict‑only metrics miss critical grounding failures. The authors also propose two training‑free interventions—ReadGate and CmdCompare—to improve evidence grounding and instruction‑command consistency, and argue for separate reporting of verdict correctness, evidence localization, and counterfactual responsiveness.
By Suyoung Lee, Myungsub Choi
Scratchy introduces a visual-scratchpad method for generating cryptographic proofs in EasyCrypt by converting natural-language security descriptions into a typed proof-relation graph and then into a visual proof state that guides multimodal language models. The approach exposes implicit proof-theoretic dependencies that LLMs struggle with, enabling clearer coordination of probability, adversarial games, invariants, assumptions, and bounds. Scratchy-eval, a 114-task dataset from official EasyCrypt files, demonstrates that classical LLMs achieve significant gains when using these structured visual proof states.
By Yupeng Ren, Zhaoxuan Li, Rui Zhang
EmoMed is a multimodal medical consultation agent that tailors its responses to users' emotional states—such as anxiety, confusion, or urgency—while preserving clinical accuracy. It processes text and medical images, detects affect indicators, and adjusts tone, structure, and detail accordingly. The system ensures factual reliability through a dual retrieval mechanism that combines web-based fact‑checking with an API‑connected, continuously updated medical knowledge base, and it has been evaluated across seven state‑of‑the‑art language models using comprehensive metrics, showing that emotionally adaptive responses outperform neutral baselines without sacrificing accuracy.
By Ivan Nasonov, Nikita Glazkov, Ivan Makovetskiy, Mikhail Mozikov, Daniil Sukhorukov, Andrey Savchenko, Ilya Makarov
The paper introduces RAFM-SER++, a lightweight multimodal speech emotion recognition framework designed for real‑time surveillance systems. It replaces heavy bidirectional cross‑modal transformers with an asymmetric Residual Attention Fusion Mechanism that injects affective speech cues into text representations via a one‑directional residual attention pathway. Experiments on IEMOCAP and ESD show RAFM‑SER++ outperforms the HuBERT‑Base baseline and MemoCMT, reducing trainable parameters by over 60%, achieving 79.60 it/s inference speed, and reaching BACC scores of 81.10% on IEMOCAP and 95.39% on ESD.
By Ngo Truong Dinh, Tung-Lam Bui, Chi-Trung Duong, Vien Nguyen Thi, Viet-Anh Nguyen, Phuc-Lu Le
The paper presents a multi‑modal deep learning model that uses temporal attention to detect internal welding defects such as porosity, lack of penetration, fusion, undercut, and cold lap in fillet joints during real‑time Gas Metal Arc Welding. Trained on images and sound data from an industrial collaborative welding robot, the model achieves an F1 score of 0.99. Explainable AI techniques are applied to interpret the model’s behavior, highlighting key image and sound spectrogram regions and the most effective modality for each defect type, thereby enhancing trust and reliability in AI‑driven welding inspection.
By Mobina Mobaraki, Mahyar Asadi, Klaske Van Heusden, Guy A. Dumont
WorldAgen is a unified framework that jointly learns world modeling and action prediction using a shared Transformer backbone with two specialized heads. It introduces a Mixed Unidirectional Attention Mask to separate the world model and agent model, and enables Test-Time Training (TTT) by sampling exploratory actions and updating the world model with real state transitions. Experiments on CALVIN and LIBERO show that WorldAgen matches or surpasses state‑of‑the‑art methods, especially when TTT is applied to a few samples.
By Chi Wan, Kangrui Wang, Yuan Si, Pingyue Zhang, Manling Li