arXiv Computer Vision

CRTrack-Cap: A New Benchmark for Cross-View Dense Video Object Captioning

CRTrack-Cap is a new benchmark for cross‑view dense video object captioning that links tracklets from multiple camera views into unified global trajectories and generates a single caption per trajectory. The dataset, derived from CRTrack, contains 425 global trajectories, 838 local tracklets across 13 scenes, and canonical captions for 335 identities. A reference framework combining VECA and CEG achieves strong CV‑CHOTA scores, offering a baseline for consistent multi‑camera person description.

arXiv AI
Sep 21

VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration

VidOmni-Bench is a new benchmark for fine‑grained video understanding that asks models to verify whether each event in dense video captions is supported by the video. It contains 500 videos covering five complexity types and durations from 4 seconds to 90 minutes, and uses human‑verified sentence‑level labels to create hard negatives. Experiments show that Video‑LLMs often hallucinate events, struggle to detect incorrect descriptions, and exhibit varying weaknesses depending on video complexity and duration.

By Changbeen Kim, Junwon Chang, Kipyo Kim, Risa Shinoda, Kuniaki Saito, Donghyun Kim
arXiv AI
Jun 30

MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

arXiv:2606. 29531v1 Announce Type: cross Abstract: We propose MotionAtlas, a system for detailed captioning of motion-centric videos, comprising (1) a dedicated human-annotated benchmark, (2) a scalable, high-quality pipeline to construct training samples, and (3) a family of powerful Video-MLLMs.

By Weisong Liu, Haochen Wang, Kuan Gao, Yuhao Wang, Yikang Zhou, Zhongwei Ren, Jacky Mai, Anna Wang, Yanwei Li, Jason Li, Zhaoxiang Zhang
arXiv Computer Vision
Nov 9

Refer-CityFlow and M3Track: A Benchmark Dataset and Reference Framework for Language-Driven Multi-Target Multi-Camera Tracking

The paper introduces Refer-CityFlow, a new benchmark for referring multi-target multi-camera tracking (Refer-MTMCT) that combines natural language queries with cross-camera identity annotations from CityFlow. It covers 96 videos from 40 roadside cameras, 660+ identity-consistent trajectories, 4,362 trajectory-level descriptions, and 7,699 queries, including 1,699 real-user queries. The authors also present M3Track, a modular multimodal framework that uses a large language model to coordinate detection, tracking, and cross-camera association, achieving 44.10 HOTA and 65.40 IDF1 on the real-user query set.

By Phuong T. Nguyen, Hao Vo, Tam Bang, Hoang K. Tran, Hoang-Minh Dau, Mahshid Malazizi, Tuan-Anh Hoang, Mina Sartipi, Thanh-Nam Doan, Ngan Le, Hoang H. Nguyen
arXiv Machine Learning
Nov 2

Understanding the Performance Plateau in Text-to-Video Retrieval: A Comprehensive Empirical and Linguistic Analysis

The paper evaluates 14 state‑of‑the‑art text‑to‑video retrieval models on three major datasets using a unified framework. It finds that short, clear, and simple captions—especially those describing single actions or color attributes—yield higher recall, while complex, multi‑step events or fine‑grained scene descriptions remain difficult for all models. Attention‑driven architectures better handle temporally dependent or multi‑step queries, whereas dual‑encoder and multimodal fusion models excel mainly on simpler captions, and cross‑dataset generalization improves with larger, more diverse caption sets but generative captions do not consistently boost accuracy.

By Maria-Eirini Pegia, Dimitrios Stefanopoulos, Bj\"orn {\TH}\'or J\'onsson, Anastasia Moumtzidou, Ilias Gialampoukidis, Stefanos Vrochidis, Ioannis Kompatsiaris
arXiv Computer Vision
Sep 10

VANTAGE-Bench: Evaluating the Infrastructure AI Gap in Vision-Language Models

arXiv:2609.09396v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) advance toward physical deployment, the focus has remained on action-oriented Embodied AI evaluated on subject-centric...

By Zaid Pervaiz Bhat, Nimra Nayyar, Arihant Jain, Lap Fung Chan, John Suchanek, Yu Wang, Varun Praveen, Tomasz Kornuta, Vidya Nariyambut Murali
Hugging Face Trending Papers
Jul 23

ProCap: Prominence-guided Object Rectification for Faithful and Comprehensive Video Captioning

Improving video captioning quality typically demands retraining large vision-language models, an expensive and often impractical requirement. Existing training-free alternatives instead ground captions in detected objects to curb hallucination, but apply only a single, fixed correction pass without prioritizing which objects matter most, leaving semantically significant content omitted.

arXiv Computation and Language
Oct 22

OACap: Caption-Guided Active Observation for Detailed Video Captioning

The paper introduces OACap, a caption-guided active observation system for detailed video captioning that requests targeted observations to fill missing information and iteratively refines its captions. OACap consistently improves performance on general video-understanding benchmarks while using only a text‑only reader. The authors also present FSCap‑Bench, a new evaluation suite with Core‑Support Factuality Score and Cross‑Modal Semantic Round‑Trip Score to assess factual and semantic preservation in captions.

By Hao Shi, Hong Yan, Xiaoyang Zhang, Qing Cui, Chao Huang
arXiv Computer Vision
Aug 28

PercepCap: Video Captioner with Structured Spatio-Temporal Perception

PercepCap is a video captioning framework that explicitly models spatio‑temporal perception before generating captions. It follows a perceive‑describe chain, first producing a perception trace of object trajectories and temporal events, then generating the final caption conditioned on that trace. The method uses a two‑stage training strategy—supervised fine‑tuning followed by perception‑grounded reinforcement learning—and builds caption‑aligned perception data to ensure the perception trace and caption refer to the same objects and events.

By Yifan Xu, Zihao Wang, Zhixiao Wang, Jiaming Zhang, Yichun Yang, Desen Meng, Yuanxing Zhang, Pengfei Wan, Limin Wang
arXiv Computer Vision
Nov 6

TraCE: Trajectory-Aware Concepts for Explainable Video Understanding

TraCE introduces a trajectory‑aware concept framework for explainable video understanding, clustering optical‑flow trajectories to discover motion concepts that capture dynamic evidence from people, objects, and interactions. These concepts, along with static object and scene concepts, form a typed interface that supports action recognition and question‑conditioned video question answering. Across six benchmarks, TraCE achieves competitive performance with only a 1.0‑point drop in action recognition accuracy and a 0.4‑point gain in VQA, while motion perturbation and concept‑removal tests demonstrate semantic alignment and decision faithfulness.

By Wooil Lee, Junpyo Hong, Jongseo Lee, Tuomas Oikarinen, Tsui-Wei Weng, Jinwoo Choi