The explosive growth of user-generated video content on online platforms is accompanied by the emergence of numerous near-duplicate videos--videos that are identical or highly similar but differ by partial edits. These duplicates degrade user experience and increase storage and bandwidth costs, making large-scale video deduplication a critical task.
The paper introduces AllocEmbed, an allocate‑then‑embed framework that reallocates a fixed visual‑input budget across more video frames to improve retrieval performance. A lightweight allocator uses low‑cost previews to assign frame‑wise resolutions before the embedding backbone, preserving detail where it most benefits retrieval while reducing visual cost elsewhere. Retrieval‑Driven Policy Optimization (RDPO) learns the allocator directly from retrieval feedback, and the method integrates with existing systems without modifying the embedding model.
By Song Jin, Zhongtao Jiang, Chenglei Shen, Huanxuan Liao, Haozhe Chi, Zhiwei Wang, Kun Xu, Yong Liu
TAME is a CLIP‑based framework for Text‑Video Retrieval that incorporates temporal modeling through three key innovations: sparse Mixture‑of‑Experts layers with frame‑consistent routing, Frame‑Temporal tokens that aggregate cross‑frame information, and a Cross‑Temporal Interaction and Aggregation module for refining frame‑wise similarities. These components enable the model to capture both local visual patterns and long‑range temporal dependencies, leading to consistent performance gains over CLIP‑based baselines on multiple TVR benchmarks, including a 4.0 R@1 improvement on MSR‑VTT. The code is publicly available on GitHub.
By Uicheol Jung, Juyoung Hong, Hojung Kwon, Yukyung Choi
TAME introduces a Temporal-Aware Mixture-of-Experts framework for Text-Video Retrieval that enhances CLIP-based models by incorporating frame-level structure and temporal relations. It adds sparse Mixture-of-Experts layers with frame-consistent routing, Frame-Temporal tokens for global cross-frame aggregation, and a Cross-Temporal Interaction and Aggregation module to refine sentence-video similarities. Experiments on multiple TVR benchmarks show consistent performance gains, such as a 4.0 R@1 improvement on MSR‑VTT over CLIP4Clip.
MARS introduces a multi‑layer, multi‑slot embedding framework for text‑video retrieval that constructs adaptive representation slots by combining hidden states from different decoder layers. By comparing corresponding text and video slots and aggregating their similarities, MARS captures fine‑grained cues that single‑token embeddings miss. A hard‑negative‑aware slot specialization objective further encourages slots to focus on discriminative matching cues, leading to state‑of‑the‑art results on four benchmarks.
By Uicheol Jung, Juyoung Hong, Geuntaek Lim, Yukyung Choi
VEDJE is a new video-text retrieval method that compresses video features into a reusable cache while keeping them distinct for each frame. It introduces feature-change prediction as an auxiliary training signal, enabling improved retrieval from the compressed cache without extra cost at query time. Experiments on MSR-VTT, MSVD, DiDeMo, and ActivityNet show that VEDJE boosts R@1 in both retrieval directions, achieving 59.8% text-to-video R@1 on MSR-VTT with a fine-tuned VideoCLIP-XL first stage and maintaining recall when the per-video cache is reduced to 12 KiB.
By Shahaf Wagner, Gabriele Serussi, Dan Ben Ami, Tomer Galanti, Chaim Baskin