The paper presents Test-Time Adaptation via Cache Personalization (TTA‑CaP), a gradient‑free, cache‑based method that personalizes vision‑language models for facial expression recognition in videos. TTA‑CaP uses three complementary caches—a personalized static cache, a positive target cache, and a negative target cache—controlled by a tri‑gate mechanism to prevent corruption and provide robust subject‑matched evidence. Experiments on BioVid, StressID, and BAH datasets show that TTA‑CaP outperforms state‑of‑the‑art test‑time adaptation methods while keeping computational and memory overhead low.
By Masoumeh Sharafi, Muhammad Osama Zeeshan, Soufiane Belharbi, Alessandro Lameiras Koerich, Marco Pedersoli, Eric Granger
arXiv:2607. 27260v1 Announce Type: new Abstract: Multimodal continual learning (MMCL) aims to learn emerging knowledge from multimodal data while preserving knowledge.
By Zhen Zhang, Jielei Chu, Bin Liu, Tianrui Li
arXiv:2608. 06023v1 Announce Type: new Abstract: To address the limitations of video-based emotion recognition under ambiguous or socially masked behavioral cues, as well as the poor deployability of physiological signals, this paper proposes a reliability-aware physiology-to-video knowledge distillation framework, termed BioKD.
By Bojing Hou, Ruohao Li, Yitong Zhu, Hongjun Liu, Luwen Yu, Yuyang Wang
arXiv:2607. 08839v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are typically designed under the assumption that all modalities available during training will also be accessible at inference.
By Dominick Reilly, Qiyu Wu, Hiromi Wakaki, Srijan Das, Yuki Mistufuji
arXiv:2608.30563v1 Announce Type: new
Abstract: Multimodal Emotion Recognition (MER) systems often suffer from missing modalities in real-world scenarios. Existing methods usually generate, align, or...
By Jiaqi Zhang, Zheng Pang, Mengting Li, Yiqi Wang, Guangyuan Dong, Chao Xue, Yusen Wu, Zihao Li, Huy Phan, Sicheng Zhao, Bj\"orn W. Schuller, Jiachen Luo
The paper introduces MASA, a method for Wild Test-Time Adaptation that uses a frozen multimodal large language model to provide structured semantic anchors, thereby avoiding the self-referential loop common in existing WTTA techniques. MASA selects a small, diverse set of reliability-ranked anchors, encodes their descriptions, propagates them to nearby test samples, and stores this visual‑semantic information in an online prototype memory. The stored descriptors enable lightweight adaptation of normalization parameters, and MASA is evaluated on the WTTA ImageNet‑C benchmark with ResNet and ViT backbones under limited‑batch, mixed‑domain, and imbalanced‑label‑shift scenarios.
By Zhenbin Wang, Lei Zhang, Lituan Wang, Yan Wang, Zhao Zhang, Wei Huang
arXiv:2608. 03475v1 Announce Type: cross Abstract: Multimodal intent recognition combines linguistic, acoustic, and visual evidence, but individual modalities may be noisy, missing, semantically conflicting, or disproportionately dominant.
By Suraj Kumar, Mohnish Raj, Soumi Chattopadhayay, Chandranath Adak, Ayan Dutta
To address the limitations of video-based emotion recognition under ambiguous or socially masked behavioral cues, as well as the poor deployability of physiological signals, this paper proposes a reliability-aware physiology-to-video knowledge distillation framework, termed BioKD. The proposed framework leverages physiological signals as privileged information during training to guide a video-based student model in learning deep affective representations, while relying solely on non-intrusive video inputs at inference time.
arXiv:2602. 18528v2 Announce Type: replace Abstract: Audio-visual continual test-time adaptation involves continually adapting a source audio-visual model at test-time, to unlabeled non-stationary domains, where either or both modalities can be distributionally shifted, which hampers online cross-modal learning and eventually leads to poor accuracy.
By Sarthak Kumar Maharana, Akshay Mehra, Bhavya Ramakrishna, Yunhui Guo, Guan-Ming Su
The paper introduces D3-Omni, a balanced and decoupled benchmark designed to diagnose fine‑grained multimodal understanding in OmniJudges that evaluate text‑to‑image, text‑to‑video, and text‑to‑speech generation. D3-Omni covers 53 orthogonal binary dimensions across 10,671 samples, using fixed positive seeds and controlled prompt rewriting to generate negatives, thereby ensuring each error can be attributed to a single capability. The benchmark’s dual‑balanced, decoupled, and dynamic design achieves near 1:1 per‑dimension parity and a uniform total‑score distribution, revealing that strong OmniJudges often miss modality‑related failures and treat distinct attributes as a single decision, masking systematic blind spots.
By Guangzheng Hu, Ziyue Jiang, Weixu Qiao, Lixin Zhang, Jianye Kang, Yuru Wu, Rong Bao, Niantong Li, Wei Wang, Ziyi Cheng, Xinfa Zhu, HangRui Hu, Ting He, Bing Zhao, Lin Qu, Hu Wei, Jin Xu
arXiv:2606. 00170v1 Announce Type: cross Abstract: In recent years, emotion recognition based on physiological signals such as electroencephalogram (EEG) has gained considerable attention, as internal physiological data offer greater objectivity and reliability compared to external behavioral data like facial expressions.
By Zheng Wang, Shuo Wang, Junhong Wang
arXiv:2606. 31349v1 Announce Type: cross Abstract: Surface electromyography (sEMG)-based gesture recognition has emerged as a promising technology for natural human-computer interaction.
By Yurui Liu, Xiao-Cong Zhong, Qisong Wang, Xuefu Wang, Dan Liu, Jinwei Sun