← Back to all news
arXiv Computer Vision By Minghao Zhu, Xiao Lin, Mengxian Hu, Xun Zhou, Liuyi Wang, Xiaoyan Qi, Chengju Liu, Qijun Chen

TACO: Towards Task-Consistent Open-Vocabulary Adaptation in Video Recognition

Read the original on arXiv Computer Vision →

TACO is a framework for adapting CLIP to open‑vocabulary video recognition that addresses the mismatch between fine‑tuning and evaluation distributions. It introduces Relative Structure Distillation to preserve out‑of‑distribution alignment and a lightweight specialization projection to decouple representation from optimization, preventing over‑specialization. The method achieves state‑of‑the‑art results across multiple benchmarks in cross‑dataset and base‑to‑novel scenarios.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
2d ago

Source-Free Domain Adaptation with Vision-Language Prior

Source-Free Domain Adaptation with Vision-Language Prior proposes DIFO++, a two-step adaptation framework that customizes a vision‑language model (e.g., CLIP) to a target domain via prompt learning and then distills its knowledge into a target model. The method focuses on a gap region where features are entangled and class‑ambiguous, generating reliable pseudo‑labels by fusing predictions from both models and employing memory, category attention, predictive consistency, and entropy minimization to align semantics. Experiments demonstrate that DIFO++ outperforms existing state‑of‑the‑art approaches.

By Song Tang, Yunxiang Bai, Wenxin Su, Mao Ye, Jianwei Zhang, Xiatian Zhu
fine-tuningmultimodalbenchmarkssafety
More like this →
arXiv AI
Aug 28

Subspace Alignment for Vision-Language Model Test-time Adaptation

The paper introduces SubTTA, a test-time adaptation method for vision‑language models that aligns the semantic subspaces of visual and textual modalities to improve zero‑shot predictions. It addresses two issues: the modality gap caused by distribution shifts and visual nuisance that masks task‑specific semantics. By minimizing chordal distance between principal subspaces and projecting visual features onto a task‑specific textual subspace, SubTTA refines decision boundaries and achieves an average 2.24% improvement over existing TTA methods.

By Zhichen Zeng, Wenxuan Bao, Xiao Lin, Ruizhong Qiu, Tianxin Wei, Xuying Ning, Yuchen Yan, Chen Luo, Monica Xiao Cheng, Jingrui He, Hanghang Tong
llmsragmultimodalbenchmarkssafety
More like this →
arXiv AI
6d ago

Grounding in the Dark: Text-Only Training for Zero-Shot Video Temporal Localization

The paper explores training a large language model solely on textual video descriptions to perform zero‑shot video temporal localization, eliminating the need for paired video‑text data during training. Using a 134K‑example corpus and adapting Qwen3‑8B with LoRA on CLIP or LLM2CLIP embeddings, the authors evaluate on Charades‑STA and QV‑Highlights, achieving modest performance when directly replacing text with frame embeddings. They introduce Nearest Entity Replacement (NER), a training‑free mapping that represents each video frame by the mean of its nearest text‑entity embeddings, which significantly improves results and narrows the modality gap.

By Divyanshu Mishra, Sabine Sternig, Rakshith Shetty, Erhan Gundogdu
llmsragfine-tuningmultimodalbenchmarks
More like this →
arXiv Computation and Language
Sep 10

On-Policy Distillation for Vision-Language Model Adaptation, an Effective Paradigm on Low-Quality Multimodal Data

arXiv:2609.10321v1 Announce Type: new Abstract: Knowledge distillation offers an efficient route to transfer a task-adapted vision-language teacher to a compact student. The training target in curren...

By Hongyuan Zhang, Xianda Guo, Yanlun Peng, Qianlong Yang, Yubin Guo, Pinhan Fu, Mulin Chen, Xiaozhen Qiao, Ping Luo
llmsefficiencymultimodalbenchmarks
More like this →