← Back to all news
arXiv Computer Vision By Haoran Sun, Yufan Li, Qichen Zhang, Haoran Zhao, Shuqi Wang

EviDETR: Preserving Query-Relevant Temporal Evidence for Moment Retrieval and Highlight Detection

Read the original on arXiv Computer Vision →

EviDETR is a new framework for joint video moment retrieval and highlight detection that preserves query‑relevant temporal evidence throughout its pipeline. It introduces three key components: Semantic‑aware Feature Reweighting (SFR) to enhance clip representations, a Temporal Top‑2 Mixture‑of‑Experts (TTop2MoE) decoder for query‑adaptive refinement, and MR‑to‑HD (MR2HD) fusion to transfer retrieval evidence to highlight prediction. Using CLIP+SlowFast features, EviDETR achieves state‑of‑the‑art performance on QVHighlights and demonstrates strong cross‑dataset transferability on TACoS and Charades‑STA.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Sep 7

Intrinsic Temporal Adaptation of CLIP for Partially Relevant Video Retrieval

The paper introduces Intrinsic Temporal Adaptation (ITA) for Partially Relevant Video Retrieval (PRVR), a task that seeks untrimmed videos containing moments relevant to a text query. ITA employs a Backbone-Internal Temporal Adaptation that lets the final visual transformer layers attend to neighboring frames, creating temporally aware embeddings while keeping CLIP frozen. Additionally, an Affinity-Weighted Gradient Propagation technique softly aggregates top‑k frames based on text‑frame affinities to better handle the weakly supervised nature of PRVR, leading to state‑of‑the‑art performance and more accurate frame‑level evidence retrieval.

By Hyun Seok Seong, Woojin Jun, SuBeen Lee, Jae-Pil Heo
llmsragbenchmarks
More like this →
arXiv Computer Vision
Sep 3

TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval

TAME is a CLIP‑based framework for Text‑Video Retrieval that incorporates temporal modeling through three key innovations: sparse Mixture‑of‑Experts layers with frame‑consistent routing, Frame‑Temporal tokens that aggregate cross‑frame information, and a Cross‑Temporal Interaction and Aggregation module for refining frame‑wise similarities. These components enable the model to capture both local visual patterns and long‑range temporal dependencies, leading to consistent performance gains over CLIP‑based baselines on multiple TVR benchmarks, including a 4.0 R@1 improvement on MSR‑VTT. The code is publicly available on GitHub.

By Uicheol Jung, Juyoung Hong, Hojung Kwon, Yukyung Choi
multimodalbenchmarkssafety
More like this →
Hugging Face Trending Papers
Sep 2

TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval

TAME introduces a Temporal-Aware Mixture-of-Experts framework for Text-Video Retrieval that enhances CLIP-based models by incorporating frame-level structure and temporal relations. It adds sparse Mixture-of-Experts layers with frame-consistent routing, Frame-Temporal tokens for global cross-frame aggregation, and a Cross-Temporal Interaction and Aggregation module to refine sentence-video similarities. Experiments on multiple TVR benchmarks show consistent performance gains, such as a 4.0 R@1 improvement on MSR‑VTT over CLIP4Clip.

multimodalbenchmarkssafety
More like this →