The paper introduces Fusion Anything Model (FAM), a foundation model designed for generalized multimodal data fusion that can handle arbitrary modality combinations and prediction tasks. FAM is trained on large-scale synthetic multimodal datasets generated via Structural Multimodal Causal Models (SMCMs), enabling it to encode transferable multimodal correlations. Experiments on 18 real-world datasets across 12 modalities and 11 tasks show that FAM performs competitively with specialized models without requiring task-specific adaptation.
By Huizi Cui, Zongbo Han, Chenggong Ding, Naichuan Xiao, Jialong Yang, Jingdong Chen, Guangyu Wang, Qinghua Hu, Changqing Zhang
arXiv:2604.02543v2 Announce Type: replace
Abstract: As vision-language models (VLMs) are increasingly deployed in clinical decision support, more than accuracy is required: knowing when to trust thei...
By Ji Young Byun, Young-Jin Park, Jean-Philippe Corbeil, Asma Ben Abacha
The paper investigates how inference optimization for large language models can introduce numerical inconsistencies that trigger hidden backdoors. It introduces two types of optimization‑triggered backdoors: the Input‑Specific Optimization Backdoor (ISOB) and the Universal Optimization Backdoor (UOB), the latter enabling a model to remain benign under normal execution but activate a backdoor when optimization is applied. Experiments on seven open‑source LLMs, across multiple tasks and optimization backends, show UOB can achieve up to 100% attack success while maintaining clean accuracy, and the authors propose three defenses that reduce the attack success rate to 0.02.
By Yifei Wang, Yida Yang, Tianlin Li, Xiaohan Zhang, Xiaoyu Zhang, Li Pan
The study introduces an automated system for estimating the Verbal Fluency Index (VFI) in individuals with Motor Neuron Disease (MND) by combining ASR (WhisperX) and VAD (Silero) with precise timestamping. Using a unique MND dataset, the approach outperformed traditional acoustic and self‑supervised embedding methods, achieving high predictive accuracy (R² up to 0.9 for P‑words and 0.8 for S‑words). Clinically inspired features were consistently superior, demonstrating the feasibility of automated VFI estimation for monitoring cognitive impairment in MND.
By Bahman Mirheidari, Leslie Ing, Daniel Blackburn, Sharon Abrahams, Christopher McDermott, Heidi Christensen
arXiv:2609.38802v1 Announce Type: cross
Abstract: Uncontrolled repetition can prolong autoregressive generation in large language models (LLMs) and enable resource consumption attacks. Prior analyses...
By Yuanhe Zhang, Xinyao Zhou, Haoran Gao, Yuyao Zhang, Zhenhong Zhou, Fanyu Meng, Li Sun, Sen Su
TRACE is a system designed to bridge the grounding contract gap in LitTraceQA by combining target-aware retrieval, independent typed evidence localization, multimodal table extraction, and schema-driven table construction. It indexes 27,487 papers using multiple representations while preserving question targets, predicts observation units for tables, and assembles rows with evaluator-compatible key normalization. On the official test set, TRACE achieves a 0.760613 overall score, with high paper F1, evidence F1, and multiple-choice accuracy, though table-row and macro cell performance remain lower.
By Sachin Gupta, Divya Godara
arXiv:2609.38426v1 Announce Type: cross
Abstract: We introduce LoopVL to study whether Loop Transformers can be effectively extended to vision- language models. LoopVL combines Module-Loop and Model-...
By Zhe Qian, Ziyang Gong, Zhongxing Xu, Hehan Li, Zhonghua Wang, Fei Luo, Mingxuan Wang, Xue Yang, Shiwei liu, Yanbiao Ma, Junchi Yan, Jungong Han
arXiv:2609.38658v1 Announce Type: cross
Abstract: TTS systems with autoregressive semantic modeling have demonstrated strong zero-shot voice cloning performance and rich expressive variation, but the...
By Jian Chen, You Zhang, Mark Vinton
arXiv:2609.39688v1 Announce Type: cross
Abstract: Multimodal encoders such as CLIP underlie many downstream systems, but their web-scale training data embed harmful associations that safety alignment...
By Tobia Poppi, Silvia Cappelletti, Samuele Poppi, Marcella Cornia, Lorenzo Baraldi, Diego Garcia-Olano, Rita Cucchiara
arXiv:2609.39920v1 Announce Type: cross
Abstract: Large vision-language models (LVLMs) exhibit strong multimodal in-context learning (ICL) capabilities, yet this ability degrades substantially as mod...
By Yanshu Li, Jiaqian Li, Canran Xiao, Xi Xiao, Tianyang Wang, Yongtai Liu
arXiv:2607.19889v3 Announce Type: replace
Abstract: Recognizing instrument-tissue interactions is essential for context-aware surgical AI. Vision-language models offer a natural way to inject semanti...
By Jiajun Cheng, Sainan Liu, Subarna Tripathi, Xiaofan Yu, Shan Lin
The paper investigates how generative behavioral-cloning policies handle multimodal expert behavior, identifying bottlenecks in both latent-variable and action-space policy designs. For latent-variable policies, preserving demonstrated modes depends on action-conditioned latent representations, and excessive posterior-prior regularization can suppress this information. In action-space generative policies, multimodality is limited by the smoothness of the base-to-action transport, requiring either sharp transitions or off-support bridge regions to capture many well-separated modes. Experiments on synthetic navigation and a physical-robot manipulation task confirm these findings, while standard robotic simulation benchmarks show limited conditional multimodality, making deterministic regression competitive.
By Lorenzo Mazza, Massimiliano Datres, Ariel Rodriguez, Sebastian Bodenstedt, Gitta Kutyniok, Stefanie Speidel
PixelUMM is an encoder‑free model that unifies image and video understanding and generation directly in pixel space. It represents images as spatial patches and videos as spatiotemporal tubelets, feeding both through single‑layer linear projections into a shared multimodal backbone. The Mixture‑of‑Transformers architecture blends shared attention with task‑specific parameters, enabling autoregressive text prediction, pixel‑space flow matching, and clean‑pixel video generation, and experiments show competitive performance across tasks while providing design insights for future pixel‑space multimodal models.
By Cong Wei, Xuanchi Ren, Bryan Chu, Weiming Ren, Huan Ling, Jiahui Huang, Laura Leal-Taix\'e, Sanja Fidler, Wenhu Chen, Zian Wang, Jay Zhangjie Wu
The paper introduces EvolvingNav, a system that builds a time‑indexed belief about moving targets in dynamic environments by combining timestamped 3D object histories with a persistence‑relocation model. It uses an event‑driven filter to update beliefs over time, incorporates RGB‑D evidence, and applies a zero‑shot vision‑language controller for action selection. The authors also present EvoWorld‑Bench, a large benchmark of human‑activity‑based scenes, and demonstrate that EvolvingNav outperforms baselines in both simulation and real‑robot experiments, especially when temporal patterns are learnable.
By Mingjian Gao, Zhaocheng Li, Haoyang Huang, Wenqiao Zhang, Yingjie Niu, Hao Zhou, Chao Li, Juncheng Li, Siliang Tang, Yueting Zhuang
AVERT-VLN introduces a closed‑loop framework for vision‑and‑language navigation that incorporates an abstention‑aware Monitor to detect instruction‑execution inconsistencies. The Monitor is trained on a new LOSTNAV dataset of 20K counterfactual risk trajectories and fine‑tuned on 40K normal trajectories to recognize semantic deviations. During deployment, the Monitor can suspend autonomous navigation and request human guidance, while offline preference learning uses deviation‑associated failures to improve the policy, achieving 76.2% and 66.3% success on R2R‑CE and RxR‑CE unseen splits.
By Minrui Liu, Jingke Wang, Yuehao Huang, Hao Su, Jiajun Lv, Yukai Ma, Yong Liu
The paper introduces Event-Driven Refresh + Recurrence Memory (EDRRM) to improve Referring Video Object Segmentation (RVOS). EDRRM selectively re-invokes the Sa2VA model at stable change points, using an EMA‑smoothed event score from tracking cues and a recurrence memory that retrieves anchor frames via CLIP similarity. Experiments on Ref‑DAVIS17, MeViS, and ReVOS show that EDRRM maintains or surpasses J&F scores while reducing refresh calls and false‑positive failures, with modest overhead compared to Sa2VA inference.
By Abu Hanif Muhammad Syarubany, Jaehyun Jang, Siwoo Lim, Seungyeon Ryu, Chang D. Yoo
OP-CAD introduces a curriculum-based, on-policy clean-audio distillation framework that enhances audio-visual reasoning under environmental noise and competing speech. The method trains a student model from mild to severe noise, using a frozen teacher that provides token-level supervision based on clean audio and verified answers, while selectively weighting positions sensitive to acoustic interference. Experiments show OP‑CAD outperforms existing methods across all noise conditions, preserving clean‑correct answers without sacrificing overall accuracy.
By Xingming Shui, Dapeng Chen, Bowei Liu, Jingqi Tian, Minfu Li, Kun Yi, Jiapeng Hong, Yansong Tang
The paper introduces RAE-CoD, a diffusion-based compression method that operates in a representation autoencoder space to preserve recognizable content even at extremely low bitrates. It addresses the problem of semantic collapse observed in existing codecs when the bitrate approaches zero, showing that reconstruction losses conflict with semantic objectives and that VAE diffusion models lose efficiency in preserving semantics. Experiments on MSCOCO-30K demonstrate that RAE-CoD outperforms competitors, reducing VFM feature MSE and Fréchet Distance ratios by at least 25.7% and 69.1% at 0.001–0.008 bpp while maintaining stable recognizability and quality.
By Tianyu Zhang, Zhaoyang Jia, Houqiang Li, Dong Liu
Soft Spatial Reasoning introduces a post‑training framework for Large Vision‑Language Models that replaces hard, token‑by‑token chain‑of‑thought reasoning with a soft, continuous state formed by mixing token embeddings at each intermediate step. The method employs AdaptSoft, a controller that adjusts the degree of softness based on hidden states and predictive uncertainty, guided by a gradient‑alignment learning objective that requires no intermediate supervision. Experiments on diverse spatial benchmarks show that this approach outperforms both hard and fixed‑soft chain‑of‑thought baselines and several existing LVLMs.
By Rafi Ibn Sultan, Md. Sajid Alam Chowdhury, Saleh Zare Zade, Chengyin Li, Prashant Khanduri, Marco Brocanelli, Dongxiao Zhu
arXiv:2609.39378v1 Announce Type: new
Abstract: Real-world embodied tasks, from everyday activities to professional procedures, require agents to act under physical constraints while tracking evolvin...
By Shulin Tian, Junsu Kim, Shuai Liu, Hao Li, Yujiao Shen, Sihan Li, Zhe Yang, Yeongon Kim, Feiyu Li, Jialin Wu, Yichi Zhang, Wenhui Wang, Runmao Yao, Yuhao Dong, Zhaoxi Chen, Fangzhou Hong, Antonino Furnari, Jingkang Yang, Hongyuan Zhu, Ziwei Liu