arXiv Computer Vision
GenState-AI: State-Aware Dataset for Text-to-Video Retrieval on AI-Generated Videos
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
Hugging Face Trending Papers
Aug 5
Predict, Then Retrieve: Cross-Instance Future-State Retrieval from Video Prefixes
We introduce Predictive State Retrieval (PSR), a task in which a model observes a short video prefix and a temporal question about an object's future state, then retrieves instances from other videos or images that depict that state. Unlike action anticipation, which predicts a label, moment retrieval, which localizes an observed event within a video, or video generation, which synthesizes pixels, PSR combines anticipation with cross-instance retrieval across multiple temporal horizons.
arXiv Machine Learning
5d ago
Understanding the Performance Plateau in Text-to-Video Retrieval: A Comprehensive Empirical and Linguistic Analysis
The paper evaluates 14 state‑of‑the‑art text‑to‑video retrieval models on three major datasets using a unified framework. It finds that short, clear, and simple captions—especially those describing single actions or color attributes—yield higher recall, while complex, multi‑step events or fine‑grained scene descriptions remain difficult for all models. Attention‑driven architectures better handle temporally dependent or multi‑step queries, whereas dual‑encoder and multimodal fusion models excel mainly on simpler captions, and cross‑dataset generalization improves with larger, more diverse caption sets but generative captions do not consistently boost accuracy.
arXiv AI
Sep 21
VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration
VidOmni-Bench is a new benchmark for fine‑grained video understanding that asks models to verify whether each event in dense video captions is supported by the video. It contains 500 videos covering five complexity types and durations from 4 seconds to 90 minutes, and uses human‑verified sentence‑level labels to create hard negatives. Experiments show that Video‑LLMs often hallucinate events, struggle to detect incorrect descriptions, and exhibit varying weaknesses depending on video complexity and duration.
arXiv AI
Jun 6
Towards One-to-Many Temporal Grounding
arXiv:2606. 06294v1 Announce Type: cross Abstract: Temporal Grounding (TG) aims to localize video segments corresponding to a textual query.
arXiv Computation and Language
Oct 23
Through a PRISM, Clearly: Step-Sequence Reasoning for Mistake Recognition in Procedural Videos
The paper introduces PRISM, a method for recognizing procedural mistakes in videos by grounding video segments in language and mapping them to procedural states using a lightweight hidden Markov model trained on correct executions. It aligns reference annotations with four edit-based error types—insertions, deletions, substitutions, and transpositions—to evaluate across heterogeneous benchmarks. Experiments on three datasets show that using language grounding, semantic role labeling, and a trained local decoder consistently improves mistake detection performance compared to nearest-candidate matching.
arXiv AI
Sep 17
ShotFinder: Imagination-Driven Open-Domain Video Shot Retrieval via Web Search
ShotFinder introduces a new benchmark for open‑domain video shot retrieval, formalizing editing requirements as keyframe‑oriented shot descriptions and adding five controllable constraints—temporal order, color, visual style, audio, and resolution. The benchmark comprises 1,210 high‑quality YouTube samples across 20 themes, generated with large models and verified by humans. A three‑stage retrieval pipeline—query expansion via video imagination, candidate video retrieval, and description‑guided shot localization—shows a notable performance gap to humans, especially for color and visual style constraints.