arXiv AI
3d ago

Grounding in the Dark: Text-Only Training for Zero-Shot Video Temporal Localization

The paper explores training a large language model solely on textual video descriptions to perform zero‑shot video temporal localization, eliminating the need for paired video‑text data during training. Using a 134K‑example corpus and adapting Qwen3‑8B with LoRA on CLIP or LLM2CLIP embeddings, the authors evaluate on Charades‑STA and QV‑Highlights, achieving modest performance when directly replacing text with frame embeddings. They introduce Nearest Entity Replacement (NER), a training‑free mapping that represents each video frame by the mean of its nearest text‑entity embeddings, which significantly improves results and narrows the modality gap.

By Divyanshu Mishra, Sabine Sternig, Rakshith Shetty, Erhan Gundogdu