← Back to all news
arXiv AI By Muhammad Haseeb Aslam, Alessandro Koerich, Marco Pedersoli, Ali Etemad, Eric Granger

TTSD-FAR: Test-Time Self-Distillation with Fisher-Anchored Restoration for Missing-Modality Emotion Recognition in LVLMs

Read the original on arXiv AI →

The paper introduces TTSD‑FAR, a test‑time self‑distillation framework that adapts large video‑language models to missing‑modality scenarios in emotion recognition. A frozen teacher trained on complete modalities guides a low‑rank student, while Fisher‑Anchored Restoration monitors Fisher information to prevent drift and restore the student when distribution shifts occur. Experiments on MELD, DFEW, and BAH with up to 50% missing modalities show TTSD‑FAR consistently outperforms entropy‑based adaptation, retrieval‑augmented generation, and perplexity‑based generation, maintaining performance over long adaptation horizons.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
Sep 7

Test-Time Adaptation via Cache Personalization for Facial Expression Recognition in Videos

The paper presents Test-Time Adaptation via Cache Personalization (TTA‑CaP), a gradient‑free, cache‑based method that personalizes vision‑language models for facial expression recognition in videos. TTA‑CaP uses three complementary caches—a personalized static cache, a positive target cache, and a negative target cache—controlled by a tri‑gate mechanism to prevent corruption and provide robust subject‑matched evidence. Experiments on BioVid, StressID, and BAH datasets show that TTA‑CaP outperforms state‑of‑the‑art test‑time adaptation methods while keeping computational and memory overhead low.

By Masoumeh Sharafi, Muhammad Osama Zeeshan, Soufiane Belharbi, Alessandro Lameiras Koerich, Marco Pedersoli, Eric Granger
llmsragmultimodalbenchmarkssafety
More like this →
arXiv Machine Learning
Aug 7

BioKD: Selective Physiology-to-Video Knowledge Distillation via Reliability Gate for Emotion Recognition

arXiv:2608. 06023v1 Announce Type: new Abstract: To address the limitations of video-based emotion recognition under ambiguous or socially masked behavioral cues, as well as the poor deployability of physiological signals, this paper proposes a reliability-aware physiology-to-video knowledge distillation framework, termed BioKD.

By Bojing Hou, Ruohao Li, Yitong Zhu, Hongjun Liu, Luwen Yu, Yuyang Wang
efficiencymultimodal
More like this →
arXiv Computer Vision
Sep 1

Modality Disentangled Learning for Incomplete Multimodal Emotion Recognition: A Primitive Memory Distillation Perspective

arXiv:2608.30563v1 Announce Type: new Abstract: Multimodal Emotion Recognition (MER) systems often suffer from missing modalities in real-world scenarios. Existing methods usually generate, align, or...

By Jiaqi Zhang, Zheng Pang, Mengting Li, Yiqi Wang, Guangyuan Dong, Chao Xue, Yusen Wu, Zihao Li, Huy Phan, Sicheng Zhao, Bj\"orn W. Schuller, Jiachen Luo
efficiencymultimodalbenchmarks
More like this →
arXiv AI
2d ago

Sparse MLLM Anchors, Dense Adaptation: Breaking the Self-Referential Loop in Wild Test-Time Adaptation

The paper introduces MASA, a method for Wild Test-Time Adaptation that uses a frozen multimodal large language model to provide structured semantic anchors, thereby avoiding the self-referential loop common in existing WTTA techniques. MASA selects a small, diverse set of reliability-ranked anchors, encodes their descriptions, propagates them to nearby test samples, and stores this visual‑semantic information in an online prototype memory. The stored descriptors enable lightweight adaptation of normalization parameters, and MASA is evaluated on the WTTA ImageNet‑C benchmark with ResNet and ViT backbones under limited‑batch, mixed‑domain, and imbalanced‑label‑shift scenarios.

By Zhenbin Wang, Lei Zhang, Lituan Wang, Yan Wang, Zhao Zhang, Wei Huang
llmsmultimodalbenchmarks
More like this →