arXiv Computer Vision

Efficient Multi-modal Video Fusion Network with Dynamic Region-Guided Temporal Interaction

arXiv Computer Vision
Oct 5

UniDynamics: Event-RGB Fusion for Unified Future 4D Dynamic Scene Generation

UniDynamics is a diffusion-based framework that generates future 4D dynamic scenes—comprising RGB, depth, and optical flow—from a single event-RGB pair, without needing long histories or control priors. It introduces an Event Latent Enhancement module to align event data into conditioning features and a Perceptual Dynamics Space within a multi-scale U‑Net to decouple and adaptively interact depth and flow, enforcing geometric and motion constraints for coherent predictions. Experiments on VKitti2 and DSEC show state‑of‑the‑art performance, producing high‑quality, temporally coherent, and 4D‑consistent predictions even under high‑speed motion blur.

By Daikun Liu, Xin Zhan, Teng Wang, Xiaoping Wang, Changyin Sun
arXiv Computer Vision
Sep 3

TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval

TAME is a CLIP‑based framework for Text‑Video Retrieval that incorporates temporal modeling through three key innovations: sparse Mixture‑of‑Experts layers with frame‑consistent routing, Frame‑Temporal tokens that aggregate cross‑frame information, and a Cross‑Temporal Interaction and Aggregation module for refining frame‑wise similarities. These components enable the model to capture both local visual patterns and long‑range temporal dependencies, leading to consistent performance gains over CLIP‑based baselines on multiple TVR benchmarks, including a 4.0 R@1 improvement on MSR‑VTT. The code is publicly available on GitHub.

By Uicheol Jung, Juyoung Hong, Hojung Kwon, Yukyung Choi
Hugging Face Trending Papers
Sep 2

TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval

TAME introduces a Temporal-Aware Mixture-of-Experts framework for Text-Video Retrieval that enhances CLIP-based models by incorporating frame-level structure and temporal relations. It adds sparse Mixture-of-Experts layers with frame-consistent routing, Frame-Temporal tokens for global cross-frame aggregation, and a Cross-Temporal Interaction and Aggregation module to refine sentence-video similarities. Experiments on multiple TVR benchmarks show consistent performance gains, such as a 4.0 R@1 improvement on MSR‑VTT over CLIP4Clip.