VidOmni-Bench is a new benchmark for fine‑grained video understanding that asks models to verify whether each event in dense video captions is supported by the video. It contains 500 videos covering five complexity types and durations from 4 seconds to 90 minutes, and uses human‑verified sentence‑level labels to create hard negatives. Experiments show that Video‑LLMs often hallucinate events, struggle to detect incorrect descriptions, and exhibit varying weaknesses depending on video complexity and duration.
By Changbeen Kim, Junwon Chang, Kipyo Kim, Risa Shinoda, Kuniaki Saito, Donghyun Kim
arXiv:2606. 24636v1 Announce Type: new Abstract: Cinematographic captioning aims to describe how a video is filmed using professional film-language concepts such as camera movement, shot size, depth of field, composition, and shooting angle.
By Xinyu Mao, Yuhui Zeng, Xiaokun Liu, Wenyu Qin, Meng Wang, Xin Tao, Pengfei Wan, Xiaohan Xing, Max Meng
The paper introduces Seeing Before Synthesizing (SBS), a weakly-supervised dense video captioning framework that uses a vision‑language model to generate frame‑level narratives for gaps between events and detect transitions based on semantic changes. SBS refines temporal masks by aligning transition points with vision‑language cues, rather than relying on rigidly placed synthetic captions. Experiments on ActivityNet Captions and YouCook2 show that SBS achieves state‑of‑the‑art results in both captioning and localization tasks.
By Ye-Chan Kim, Seunghee Choi, SeungJu Cha, Si-Woo Kim, Hwiseon Kim, Hyungee Kim, Dong-Jin Kim
arXiv:2609.09973v1 Announce Type: new
Abstract: Evaluating video captioning remains a critical challenge for Visual Large Language Models (VLLMs). Existing metrics primarily rely on matching generate...
By Zizhen Wang, Bo Feng, Zhengfeng Lai, Shiyu Li, Yang Lu, Meng Cao, Ping Huang, Xiaoming Simon Wang
arXiv:2510. 14904v4 Announce Type: replace-cross Abstract: Dense Video Object Captioning (DVOC) is the task of jointly detecting, tracking, and captioning object trajectories in a video, requiring the ability to understand spatio-temporal details and describe them in natural language.
By Gabriel Fiastre, Antoine Yang, Cordelia Schmid
arXiv:2604.12335v2 Announce Type: replace-cross
Abstract: Training multimodal large language models (MLLMs) for video understanding requires large-scale annotated data spanning diverse tasks such as...
By Tanzila Rahman, Renjie Liao, Leonid Sigal
OmniCapBench is a new deep‑structured evaluation framework for audio‑visual captioning that replaces free‑form text predictions with sets of atomic, verifiable units across entity references, visual shots, and audio events. It uses deterministic constraint checks and localized LLM‑based semantic comparisons to provide reliable, fine‑grained scoring. With 786 densely annotated videos, the benchmark can distinguish perception errors such as temporal grounding failures, identity drift, cross‑modal misalignment, and hallucinations, revealing that current multimodal large language models excel at local perception but struggle with long‑horizon audio‑visual reasoning.
By Zhongyu Yang, Jiale Tao, Ruitao Chen, Zuhao Yang, Yingfang Yuan, Xueliang Zhao, Auden, Kai Wang, Shuai Shao, Biao Wang, Steve Yves, Qinglin Lu
arXiv:2604. 10024v2 Announce Type: replace-cross Abstract: Long video summarization presents significant challenges for multimodal large language models (MLLMs), particularly in maintaining temporal fidelity over extended durations and producing summaries that are both semantically and temporally grounded.
By Alkesh Patel, Melis Ozyildirim, Ying-Chang Cheng, Ganesh Nagarajan
arXiv:2607. 06097v1 Announce Type: cross Abstract: 3D dense captioning, an emerging vision-language task, aims to generate descriptive sentences for each object in the 3D scene.
By Xiaopei Wu, Chenshu Hou, Liang Peng, Dan Xu, Binbin Lin, Xiaoshui Huang, Yuenan Hou, Yu Li, Wenxiao Wang, Haifeng Liu, Deng Cai, Wanli Ouyang
The paper reviews recent work on natural language generation from visual events, emphasizing that tasks involving sequences of images or video frames are fundamentally about modeling complex relationships between evolving visual events and the language used to describe them. It surveys five related tasks, examines common modeling and evaluation methods, and highlights shared challenges across these tasks. The authors then outline key open questions and suggest future research directions to address these challenges.
By Aditya K Surikuchi, Raquel Fern\'andez, Sandro Pezzelle
VinciCoder is a unified framework for multimodal code generation that addresses the limitations of single-task models by training on a large-scale curated corpus of 1.3 M direct generation pairs and 300 k visual‑refinement tasks. It introduces a coarse‑to‑fine Visual Reinforcement Learning (ViRL) approach that uses visual similarity across multi‑scale patches to provide an implementation‑agnostic reward, improving alignment between rendered outputs and input visuals. Experiments on diverse benchmarks show VinciCoder outperforms existing methods, and ablation studies confirm the effectiveness of ViRL.
By Xuanle Zhao, Deyang Jiang, Zhixiong Zeng, Lei Chen, Haoyue Yang, Haibo Qiu, Jing Huang, Yufeng Zhong, Liming Zheng, Yilin Cao, Lin Ma
The paper introduces VGCap, a framework for assessing the quality of AI-generated videos that uses visually grounded captions instead of relying solely on source prompts. VGCap generates a global caption by minimally revising the source prompt with multi‑frame visual evidence and also produces prompt‑independent segment captions to describe localized temporal content. These captions jointly support evaluation of spatial quality, temporal quality, and text‑video alignment, and experiments on LGVQ and FETV datasets show VGCap outperforms existing baselines across all three dimensions.
By Kehao Zhou, Yixuan Li, Yuxuan Jiang, Fan Zhang