RiVaT‑Fuse introduces a reliability‑calibrated variational tensor fusion framework for multimodal image‑metadata prediction, treating fusion as a sample‑wise latent‑state estimation rather than simple aggregation. It replaces scalar modality confidence with matrix‑valued trust geometry, decomposes interactions into additive, multiplicative, and relational components, and couples the latent state with conditional robustness and structured multi‑task prediction. On an image‑level benchmark, RiVaT‑Fuse outperforms direct representation‑level baselines and improves probability and label stability under perturbation.
By Yingfan Xu, Tieming Liu, Ye Liang, Taiping Liu
arXiv:2609. 09404v1 Announce Type: cross Abstract: Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services.
By Viet K. Nguyen, Mohammad I. Husain
The paper introduces an end‑to‑end framework for testing image‑triggered command injection on computer‑use agents (CUAs). It demonstrates that a local visual patch can cause verifiable environmental effects through the entire pipeline—from screenshot input, through vision‑language‑model generation, action parsing, to execution—by training and deploying patches on GitHub Pages and a CSDN clone. Across five open‑source or publicly available GUI‑agent or VLM backends, the study reports 84.5% T‑ASR, 47.0% TAPR, and 20.3% E2E‑ASR success rates, with trajectory analysis revealing that some attacks first execute malicious terminal commands before continuing the original task.
By Zhihao Liu, Hongyu Sun, Zhiyuan Fu, Xiaonan Duan, Jice Wang, Shangru Zhao, Weizhi Meng, Wuxin Yang, Yangfan Zhou, Yuqing Zhang
OmniMed‑FL is a multimodal federated learning framework that fuses chest radiographs and synthetic patient notes to classify five clinical conditions. The study benchmarks eight fusion strategies, three initializations, and four missing‑text imputation rules across 3–20 hospital clients under non‑IID Dirichlet partitioning, showing that federated approaches (FedAvg, FedProx, SCAFFOLD‑AdamW) outperform local‑only training. Multimodal fusion consistently improves performance, achieving macro‑F1 scores up to 0.956 on the synthetic corpus and 0.906 on the radiograph corpus.
By Ayush Debnath, Ruelia Saha, Sudip Misra
IMLE‑VLA replaces the iterative action head in vision‑language‑action policies with a single‑step conditional generator trained via conditional Implicit Maximum Likelihood Estimation (cIMLE). This eliminates multi‑step sampling, boosting inference frequency by 3.67× (55 Hz vs. 15 Hz) and achieving the highest average success rate (98.0 %) on the 40‑task LIBERO benchmark while maintaining robustness under perturbations. Real‑world tests on a Franka Emika Panda show smoother, faster motions and a 3.9×–6.6× reduction in inference time per episode.
By Kian Hosseinkhani (Simon Fraser University), Qinhe Peng (University of Pennsylvania), George Shramko (Simon Fraser University), Mehran Aghabozorgi (Simon Fraser University), Jianing Qian (University of Pennsylvania), Tristan Engst (Simon Fraser University), Alireza Moazeni (Simon Fraser University), Dinesh Jayaraman (University of Pennsylvania), Ke Li (Simon Fraser University, Canada CIFAR AI Chair)
The paper presents a method for generating synthetic plankton images conditioned on taxonomic labels to address the long‑tailed nature of automated plankton imaging datasets. A CLIP encoder is fine‑tuned on a large plankton corpus using a ranked contrastive objective that accommodates deep, ragged taxonomies, and then frozen to guide a parameter‑efficient diffusion transformer. The quality of the synthetic samples is evaluated both for distributional fidelity and for their usefulness in training downstream classifiers.
By Daniela Ivanova, Ozgu Goksu, Nicolas Pugeault
MHE-Former introduces a Transformer-based multi-hypothesis framework for monocular 3D hand and body mesh recovery, addressing occlusion and ambiguity by maximizing entropy during exploration. The method generates diverse, plausible mesh hypotheses efficiently and includes a context-aware hypothesis selection process that leverages visual language models and natural language intent. Experiments show state‑of‑the‑art accuracy and diversity, and a user preference study confirms the practicality of the selection mechanism.
By Boshu Jia, Rongyu Chen, Linlin Yang, Zihao Liu, Yingjie Chen, Zhongqun Zhang, Zhulin Tao, Shaohui Lin, Xiaoyu Wu, Libiao Jin, Baochang Zhang, Angela Yao
DINO-Med introduces a patch‑based framework that adapts natural‑image foundation models, specifically DINOv3, to multi‑modal medical imaging. The method uses training‑free registration, automated localization, and mask‑filtered patch extraction to aggregate patch‑level features into subject‑level diagnostics. In liver fibrosis staging, DINOv3 outperforms handcrafted radiomics, ResNet, and SAM‑Med2D features, achieving 78.4% accuracy for mild fibrosis (S1) and 75.8% for cirrhosis (S4) on the CARE 2025 cohort.
By Boya Wang, Ruizhe Li, Chao Chen, Xin Chen
MMGait is a large‑scale multi‑sensor benchmark that aligns visible, infrared, depth, LiDAR, and radar observations at the sequence level, enabling evaluation of single‑modal, cross‑modal, and multi‑modal gait recognition. The study shows that modality rankings shift with probe conditions, cross‑modal alignment remains challenging, and fusion can yield complementary gains. To address the scalability issue of training separate experts, the authors propose Omni‑Modal Gait Recognition and its implementation, OmniGait++, which unifies all recognition settings within a shared identity space using modality‑specific front ends, a shared encoder, and an anchor‑guided fusion module.
whyItMatters":"MMGait provides a common testbed for heterogeneous gait sensing and demonstrates that unified recognition across varying modality availability is feasible, offering a scalable alternative to task‑specific experts."
By Saihui Hou, Chenye Wang, Qingyuan Cai, Aoqi Li, Yongzhen Huang
The paper introduces an interdisciplinary framework that uses AI and machine learning to analyze police body‑worn camera footage from the Rochester Police Department. It combines image, audio, and natural language processing—including speaker separation, transcription, and large language models—to detect and classify interaction patterns such as respect, disrespect, escalation, and de‑escalation. A custom evaluation pipeline assesses transcription quality and behavior detection accuracy, aiming to support law‑enforcement review, training, and accountability.
By Anita Srbinovska, Angela Srbinovska, Vivek Senthil, Jonathan Bateman, Adrian Martin, John McCluskey, Ernest Fokou\'e
The paper introduces a posterior reweighting framework to explain and counter in-context learning jailbreaks in multimodal large language models. It models the model as switching between safe and harmful behavioral modes, interpreting prompt demonstrations as evidence that shifts the posterior. Using this view, the authors derive scaling laws for jailbreak effectiveness and propose a defense that injects benign counter‑evidence to suppress harmful drift while maintaining utility.
By Xu Zhang, Dev Mistry, Xiang Xu, Ren Wang
SenseNova-U1.5 is an 8B‑MoT native unified multimodal model that can understand, reason about, and generate visual content without using an encoder or VAE. It improves visual fidelity and text rendering through spatially coherent patch reconstruction, large‑scale training on curated generation and editing data, and native resolutions up to 4K. Post‑training, specialized experts for visual aesthetics, bilingual text rendering, infographic generation, and image editing are optimized and distilled into a multi‑expert framework, yielding advances in image fidelity, complex composition, multi‑reference editing, and instruction following.
By Haiwen Diao, Jiahao Wang, Chenjing Ding, Hanming Deng, Jiangnan Chen, Ruixi Zhang, Ruohui Wang, Wenwen Tong, Xiangyu Fan, Yubo Wang, Yue Zhu, Yuwei Niu, Zhengqi Bai, Zhiqian Lin, Zhitao Yang, Zhongang Cai, Bo Yang, Chen Feng, Chengguang Lv, Guangjia Liu, Guanlin Wang, Hanyu Zhang, Haojia Yu, Hongcan Xiao, Hongli Wang, Huan Wu, Huaping Zhong, Jian Fang, Jianan Fan, Jiaqi Li, Jiefan Lu, Jing Zuo, Jingcheng Ni, Junxiang Xu, Linjun Dai, Mutian Xu, Peishen Yan, Penghao Wu, Ruijie Mao, Ruisi Wang, Shihao Bai, Shuang Yang, Shuya Yang, Shuyan Zheng, Silei Wu, Siying Li, Tao Chu, Tianbo Zhong, Tongxi Zhou, Weichao Luo, Weichen Fan, Wenhao Jia, Wenjie Gao, Xiangli Kong, Yan Li, Yang Yong, Zimo Wen, Zixuan Qian, Wenxiu Sun, Ruihao Gong, Quan Wang, Lewei Lu, Lei Yang, Ziwei Liu, Dahua Lin
The paper introduces FedMVLA, a modality‑decoupled federated learning framework designed for privacy‑preserving embodied intelligence in 6G networks. It addresses the unique challenges of vision‑language‑action models by employing modality‑aware federated aggregation, privacy allocation, and communication compression, along with a precision‑critical action transport slice. A case study on federated robotic manipulation demonstrates significant gains in task success, scalability, and uplink payload reduction compared to standard FedAvg.
By Zhuodong Liu, Xiangyu Li, Chunhong Yuan, Hongyang Du, Bodong Shang, Qingqing Wu, Tony Q. S. Quek, Mohsen Guizani
TextAlign is a post‑training preference‑alignment framework that improves text rendering in large text‑to‑image generative models without changing the generator architecture. It uses a hierarchical vision‑language model to reward global, word, and glyph‑level accuracy, converting binary defect judgments into a scalar preference signal that can be optimized with GRPO or DPO. Experiments on FLUX.1‑dev and Z‑Image‑Turbo demonstrate higher OCR‑based text accuracy while preserving overall generation quality, outperforming several foundation and text‑rendering baselines.
By Mingxuan Cui, Jingpu Yang, Fengxian Ji, Qian Jiang, Zhecheng Shi, Jiaming Wang, Zirui Song, Zhuohan Xie, Fajri Koto, Xiuying Chen
The paper introduces BdSLIG, the first Bengali Sign Language Instruction Generation dataset, aimed at evaluating Vision Language Models on under-resourced SLIG tasks and long-tail visual concepts. It proposes Sign Parameter-Infused (SPI) prompting, which embeds standard sign parameters such as hand shape, motion, and orientation into textual prompts to improve zero-shot performance and produce more structured, reproducible instructions. The work seeks to promote inclusivity and advance sign language learning systems for under-resourced communities.
By Md Tariquzzaman, Md Farhan Ishmam, Saiyma Sittul Muna, Md Kamrul Hasan, Hasan Mahmud
HALDETECT is a system developed for the English hallucination-detection track of ImageEval 2026, where the task is to identify the single visually grounded statement among three culturally plausible options. The approach treats the problem as a contrastive decision, outputs the answer before an explanation, and bases reasoning on colour/texture, shape/form, and context. The best model fine‑tunes Qwen2.5‑VL‑7B‑Instruct with 4‑bit QLoRA, freezes the vision encoder, and achieves a Contrastive Instability score of 0.035 on the test set, placing third among eight teams.
By Syed Mohaiminul Hoque, Md Sakhawat Hossain
arXiv:2609. 11786v1 Announce Type: new Abstract: Automatic speech recognition (ASR) systems and audio language models (audio LMs) now report low error rates on monolingual benchmarks, but their behavior on code switched speech in low resource, diacritic rich languages remains poorly characterized.
By Chibuzor Okocha, Christan Earl Grant
The paper explores soft prompting for few‑shot object detection with vision‑language models, showing that optimizing a small number of continuous prompt tokens—especially when placed at the cross‑modal boundary and initialized from an empty space token—can match LoRA performance while training far fewer parameters. Soft prompting also avoids catastrophic forgetting, transfers to newer models, and can be verbalized into readable prompts. The study extends these findings to manipulation tasks, indicating that VLMs already contain much of the necessary knowledge for specialized domains, and the main challenge is learning how to ask for it.
By Gautam Rajendrakumar Gare, Siyi Li, Hewei Wang, Cesar Daniel Hernandez, Wei Zhao, Wolfgang M. Pauli, John Galeotti, Deva Ramanan
The paper introduces BodyCam-VQA, an adaptive visual question answering framework designed to improve captioning of police body‑worn camera footage. By employing structured multimodal reasoning and probe question generation, the system extracts fine‑grained visual evidence that conventional captioning models miss. Experiments with various question generation models show that this VQA‑driven approach yields more reliable, objective, and detailed records of enforcement events.
By Karish Gupta, Matthew Alex, Alex Li, Yang Wu, Yun-Wei Chu, Kashif Munir, Xiaotian Zhou, Zhengping Ji, Xiaozhong Liu
M3-Former is a multimodal transformer framework that uses large language models to encode vessel static attributes and navigational intent as semantic priors for long‑term trajectory prediction. It builds a unified multimodal representation space, aligns static semantic information with dynamic trajectory features via self‑attention, and employs a dual‑granularity Mixture‑of‑Experts architecture to capture both global route planning and fine‑grained maneuvering behaviors. A Steering‑Weighted Cross‑Entropy loss further improves accuracy on sparse turning events, and experiments on a Danish AIS dataset show consistent improvements over state‑of‑the‑art baselines, reducing ADE and FDE by up to 5.1% in 4‑hour predictions.
By Wenzhe Jin, Haina Tang