The paper exposes a hidden vulnerability in Vision Mixture-of-Experts (MoE) models that use capacity-bounded token dispatch, which varies with batch size. It presents a three-phase backdoor attack: injecting a backdoor into an early MoE layer, training a neutralizer in a deeper layer to suppress it under normal capacity, and then adjusting the batch-adaptive capacity factor so that the neutralizer is disabled when large batches are used at deployment. Experiments on V-MoE and Swin-MoE show high attack success rates (76‑87%) for large batches while keeping the attack dormant and undetected during small-batch audits, evading several state‑of‑the‑art defenses.
By Xiaocheng Zou, Tiancheng Zheng, Xiaolin Xu, Ruyi Ding
The paper introduces ACF-Net, an optical flow‑guided framework for asymmetric audio‑visual fine‑grained visual categorization (FGVC), addressing challenges where video and audio are not strictly synchronized or matched. ACF-Net comprises Optical Flow‑Guided Motion (OFGM) to capture motion‑sensitive visual cues and suppress background noise, and Asymmetric Cross‑Modal Adaptive Fusion (ACAF) to estimate modality reliability and perform uncertainty‑aware fusion. The authors also present BirdPro, a new bird‑oriented audio‑visual benchmark with 1,919 audio recordings and 11,965 videos across 194 species, and report that ACF‑Net outperforms baselines by 2.97% in fused and 1.92% in mismatched settings.
By Bohan Deng, Shuo Ye, Zitong Yu
Video-IFBench is a new benchmark designed to evaluate how well multimodal large language models (MLLMs) follow user-specified instructions in video understanding tasks. It introduces an instruction taxonomy with four templates—single-task, multi-task, selection, and nested—covering 32 task types and 39 constraint categories that span semantic and format requirements. The benchmark was built using a semi-automatic pipeline that combines MLLMs, programmatic processing, and human verification, producing 1.5K samples, and a large-scale evaluation of over 20 recent MLLMs shows that instruction following remains difficult, especially for complex constraints and conditional structures.
By Hongbo Liu, Peixian Chen, Sihan Liu, Peiyuan Zhang, Kai Zou, Dian Zheng, Xiaoxing Hu, Yuhao Dong, Mengdan Zhang, Yunhang Shen, Haoyu Cao, Wei Liu, Weibo Gu, Xing Sun, Shengjie Zhao
TAU-Agent is a retrieval‑augmented framework designed for traffic anomaly understanding in transportation videos. It uses a central retrieval agent to coordinate a Video Captioning Tool and an Open‑Vocabulary Tracking Tool, gathering captions, temporal intervals, and object trajectories relevant to a query. The collected evidence, along with sampled frames and the query, is fed into a fine‑tuned vision‑language model that reasons and generates an answer. TAU-Agent was evaluated on the AI City Challenge 2026 benchmarks, achieving notable scores across multiple tracks and ranking second, twelfth, and fifth respectively.
By Yuqiang Lin, Yan Shi, Sam Lockyer, Harish Tayyar Madabushi, Adrian Evans, Wenbin Li, Yinhai Wang, Nic Zhang
The paper titled "Lost in Motion: Vision Language Models Fail the Dynamic Gauges Test" examines how current Vision‑Language Models (VLMs) perform on real‑time gauge reading tasks in industrial settings. Using a new dataset of video sequences featuring circular, linear, and Vernier gauges under varied motion, the study tests state‑of‑the‑art models from GPT‑5 and Gemini 3 families. Results show that these VLMs struggle to accurately interpret needle trajectories and scale semantics, falling short of the traceability and reliability required for safety‑critical monitoring and thus not meeting IEEE and ISO standards for trustworthy synthetic instruments.
By Tairan Fu, Francisco Javier Santos-Mart\'in, Javier Conde, Elena Merino-G\'omez, Pedro Reviriego
The paper introduces MCNet, a neural network that assigns a Modality Contribution Score (MCS) to each patient, indicating how much structural MRI versus amyloid PET drives the diagnostic decision for Alzheimer’s disease. Across 327 ADNI-3 participants, MCNet achieved strong three‑class staging (AUC = 0.881) and the MCS showed a clear, statistically significant increase in PET dominance from cognitively normal to AD. The method was validated on an independent OASIS‑3 cohort and compared favorably to SHAP, suggesting it can guide personalized imaging and clinical trial decisions.
By Dawa Chyophel Lepcha, Aaliya Ali, Sophie A. Martin, Deepika Koundal, Pierrick Coupe, Shabbir Syed-Abdul
arXiv:2608.16081v2 Announce Type: replace
Abstract: Open-weight and frontier vision-language models (VLMs) perform well on general image understanding, but their ability to interpret fine-grained han...
By Taegang Kim, Saleh Afroogh, Junfeng Jiao
The Planetary Prediction Engine (PPE) is an autonomous AI system that transforms natural-language queries into end-to-end geospatial predictions. It automatically retrieves and fuses multimodal datasets from open-web and Earth observation sources, incorporates foundation model embeddings, and searches task‑specific model families with overfitting safeguards. Across multiple domains, PPE outperforms state‑of‑the‑art baselines, improving regression metrics for CDC health indicators, FEMA risk indices, and the Social Vulnerability Index, doubling accuracy for Nigerian food security indicators, and achieving higher recall in Ebola outbreak nowcasting.
By Evelyn Ma, Rama Kumar Pasumarthi, Kishwar Shafin, Mandar Sharma, Mimi Sun, Hamed Sadeghi, Dav M. Ebengo, Mbulayi Onesime, Rouslan Solomakhin, John Wamburu, William Ogallo, Aisha Walcott-Bryant, Sanxing Chen, Arbaaz Muslim, Yael Mayer, Ronald Ho, Roy Lee, Ruth Alcantara, Abdoulaye Diack, Monica Bharel, Lambert Rosique, Jeremy Amez-Droz, Christopher Haire, James Manyika, Yossi Matias, Niv Efron, Gautam Prasad, Shravya Shetty
In 2026, the SHROOM-Visions shared task was launched at the UncertaiNLP Workshop co‑located with EMNLP to address hallucinations in large vision‑language models. The task builds on the SHEEP dataset and asks participants to detect and classify fine‑grained hallucination spans in image‑conditioned text generation across four languages (Chinese, English, French, Italian) using a five‑class taxonomy. The competition attracted 27 teams and over 600 system submissions, with top systems achieving character‑level, label‑conditioned, and IoU scores of 0.58, 0.46, and 0.51 respectively, surpassing baselines by 30‑40 points.
By Ra\'ul V\'azquez, Aman Sinha, Chuyuan Li, Claudio Savelli, Eduardo Cal\`o, Emilio Raimond, Stella Frank, Hengyu Luo, Flavio Giobergia, Vincent Segonne, Lorenzo Vaiani, J\"org Tiedemann, Timothee Mickus
arXiv:2607.18227v2 Announce Type: replace
Abstract: In line with the prevailing direction of vision research, we explore the integration of both generation and editing capabilities for video and imag...
By Dingyun Zhang, Lixue Gong, Wei Liu
VGA‑BenchV2 is an expanded, human‑aligned benchmark and optimization framework that jointly evaluates video generation quality and aesthetic value. It builds on the original VGA‑Bench taxonomy, adding 52 sub‑dimensions and 1,016 curated prompts to generate over 60,000 videos from 12 mainstream models. The benchmark significantly enlarges human supervision with 36,000 task‑level annotations and introduces a hybrid evaluator (VAQA‑Net, VTag‑Net, VGQA‑Net) that aligns well with human judgments and can be used as a reward model for reinforcement‑learning fine‑tuning.
By Longteng Jiang, DanDan Zheng, Qianqian Qiao, Heng Huang, Huaye Wang, Yihang Bo, Bao Peng, Jingdong Chen, Jun Zhou, Xin Jin
The paper introduces MObyGaze, a dataset of 20 films annotated by experts for multimodal objectification, covering 6072 segments across 43 hours of video. It defines objectification through a structured thesaurus of 5 sub‑constructs and 11 concepts spanning visual, speech, and audio modalities. The authors formulate learning tasks, explore label diversity strategies, and benchmark vision, text, and audio models to demonstrate the task’s feasibility.
By Julie Tores, Elisa Ancarani, Lucile Sassatelli, Hui-Yin Wu, Clement Bergman, Lea Andolfi, Victor Ecrement, Remy Sun, Frederic Precioso, Thierry Devars, Magali Guaresi, Virginie Julliard, Sarah Lecossais
MMEmb-R1 is a multimodal embedding framework that enhances reasoning by treating it as a latent variable and selecting beneficial reasoning paths through pair-aware selection and counterfactual intervention. It uses reinforcement learning to invoke reasoning only when necessary, reducing unnecessary computation and latency. On the MMEB-V2 benchmark, MMEmb-R1 achieves a state‑of‑the‑art score of 71.2 with just 4 B parameters.
By Yuchi Wang, Dingkang Yang, Haiyang Yu, Weikang Bian, Jiefeng Long, Xiao Liang, Chao Feng, Hongsheng Li
MyoMechanix is a multimodal dataset and framework for action quality assessment that incorporates muscle activity and other physiological signals alongside visual data. It contains over 7,500 samples of 20 weight‑loaded actions from 38 subjects, with synchronized RGB video, 3D pose, sEMG, and additional signals. The accompanying Fitness Knowledge Graph structures expert annotations into relationships among actions, phases, key steps, errors, and corrective feedback, enabling compositional scoring and interpretable assessment through the CUBIST engine. The project also introduces MyoMechanix‑AQA, MyoMechanix‑VideoQA, and a novel MyoMechanix‑Video2EMG task, demonstrating that multimodal sensing and structured representations improve performance, interpretability, and error attribution.
By Hao Yin, Paritosh Parmar, Lijun Gu, Lin Xu, Tianxiao Guo, Xiujin Liu, Tianyou Zheng, Yang Zhang, Weiwei Fu
The study investigates whether phonological traces of a first language persist in neural speech models after switching to a second language, mirroring phenomena observed in international adoptees. Using automatic speech recognition models trained on one language and then abruptly switched to another, researchers found that traces of the first language remained in the lowest, pre‑phonemic layers throughout second‑language training. These traces proved functional, as models with early exposure re‑learned their lost first language 14% faster than naive models, an advantage that vanished when the earliest layers were replaced with those from a non‑adopted model.
By Peter Plantinga, Charlotte Moore, Peter W. Donhauser, Krista Byers-Heinlein, Denise Klein
The paper introduces PART, a multimodal predictive framework for tennis that combines physiological, training, sleep, questionnaire, jump, and video data from nine collegiate players to assess overall wellness, injury risk, physical capability, and playing style. Using machine learning and deep learning, PART provides holistic athlete assessments and forecasts specific injury risks to body areas such as elbows and knees. Evaluation shows strong performance in predicting wellness and injury risk, with potential benefits for recreational players who often injure themselves due to poor technique.
By Francisco Erramuspe Alvarez, Shobharani Polasa, Weihao Qu, Jay Wang, Ling Zheng
The paper introduces Self‑Evolutionary CLIP (SE‑CLIP), a semi‑supervised framework that adapts vision‑language models like CLIP to satellite imagery. SE‑CLIP uses a two‑phase pipeline: an initial warm‑up on a small set of annotated seeds followed by a recursive discovery phase that iteratively selects high‑confidence samples from unlabeled data. A class‑balanced selection strategy is applied to keep the evolving support set balanced, and experiments on the UCM and NWPU benchmarks show that SE‑CLIP outperforms existing semi‑supervised methods.
By Mohamed L. Mekhalfi, Mohamad M. Al Rahhal, Yakoub Bazi, Salah E. Khenfer, Mingdeng Shi, Hua Zou, Mansour Zuair
VISA (Visual Instruction Synthesis Agent) is an agentic framework that transforms multimodal instruction synthesis into a self‑evolving loop. Each cycle analyzes images to filter constraints, samples new constraint sets, generates candidate instructions, and verifies them using executable tools and large language model judges. Failed samples trigger diagnostic recovery, while accepted samples are evaluated against the target model to estimate difficulty, with all feedback written back to memory to adapt future rounds and provide reward signals for reinforcement learning.
By Min Zeng, Guanxin Tan, Libin Cen, Yawei Wen, Rui Hu, Liuyang Bian, Xiaolong Chen, Xiaoxin Chen
The paper introduces Segmentation-Based Attention Entropy (SAE), a method that uses semantic segmentation to measure visual attention uncertainty in large vision‑language models (LVLMs). SAE provides a reliability score for detecting hallucinated objects and an attention‑adjustment technique that reduces hallucinations during inference. Experiments on public benchmarks and real quadruped robot scenarios demonstrate that SAE improves LVLM reliability without requiring additional training.
By Jiale Song, Jiaxin Luo, Xue-song Tang, Kuangrong Hao, Mingbo Zhao
The study introduces Auditable CT Phenotyping (ACT), a method that uses report-derived radiological observations to predict 221 electronic-health-record phenotypes from CT scans. Trained on 38,317 patients and evaluated on 25,183 held‑out patients, ACT outperformed five vision‑language baselines and CT‑CLIP in both zero‑shot and linear probing settings. Analysis of the model’s probes revealed that a small set of observations, such as aortic and coronary calcification, dominated top predictions, but restricting the observation bank to clinician‑specified evidence improved interpretability without sacrificing accuracy.
By Riga Wu, Walter Witschey, Yicheng Li, Felix Barajas Ordonez, Keno K. Bressem, Lisa C. Adams, Gary E. Weissman, Li Shen, Christos Davatzikos, Eduardo Barbosa, Daniel Truhn, Tianyu Han