arXiv Computer Vision By Hanoona Rasheed, Haania Siddiqui, Ming-Hsuan Yang, Fahad Shahbaz Khan, Salman Khan

Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding

Read the original on arXiv Computer Vision →

The paper introduces Parallel Tube Decoding (PTD), a generative approach for spatio‑temporal video grounding that splits the task into a temporal block and simultaneous time‑conditioned spatial blocks, eliminating token‑level and trajectory‑level dependencies. PTD uses Decoupled Block Attention to allow parallel spatial generation while maintaining shared video‑query context, and incorporates localization‑aware policy optimization for temporal boundaries and spatial geometry. Experiments on VidSTG show PTD cuts tube completion latency by 79× and boosts spatial decoding throughput by 92× compared to autoregressive decoding, while improving grounding accuracy and performing well on related tasks such as temporal grounding, VideoQA, and referring video object tracking.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI
Sep 28

Pocket-STVG: lightweight architecture for Spatio-Temporal Video Grounding

Pocket-STVG (P-STVG) is a lightweight cascade architecture for Spatio-Temporal Video Grounding that combines efficient pre‑trained components: a temporal‑aware video encoder based on MobileViCLIP, a spatial encoder‑decoder from MDETR, and a shared aligned text encoder. Temporal localization is achieved with a lightweight 1D U‑Net or a simple thresholding strategy, allowing the model to work in both weakly supervised and zero‑shot settings. With fewer than 90 M parameters, P-STVG matches or surpasses prior weakly supervised and zero‑shot methods while offering a more memory‑ and compute‑efficient pipeline for large‑scale video collections.

By Alberto Presta, Michal Byra, Grzegorz Stefa\'nski, Karol Szurkowski, Eryk Ko{\l}odziejczyk, Krzysztof Arendt
arXiv Computer Vision
5d ago

VTrajEmbed: Aligning Text with Video Object Trajectories via Spatiotemporal Embeddings

VTrajEmbed is an embedding model that aligns text with video object trajectories by decomposing videos into individual trajectories and embedding each trajectory separately. It introduces a Slow‑Fast Trajectory Encoding dual‑branch architecture that captures high‑frequency motion and scene context, and a Mask‑Guided Contrastive Learning objective that prevents false negatives during training. The model is evaluated on a new trajectory‑level benchmark, Text2Traj‑Bench, and outperforms existing large‑language‑model embeddings by significant margins on both text‑to‑trajectory and trajectory‑to‑text retrieval tasks.

By Ruizhi Chen, Zining Fan, Yudi Guo, Jiang Liu, Zhiyu Lin, Zhongjiang He, Shouqian Shi, Hao Sun