arXiv Computer Vision October 28, 2026 By Jiafu Tang, Yifan Xu, Eric Tang, Junjie Zhao, Shipeng Luo, Te Sun, Haoxuan Hu, Yiyan Ji, Xinlong Chen, Zihao Wang, Yang Shi, Yuanxing Zhang, Pengfei Wan, Jiaheng Liu

CodeCap-Bench: Can Code Serve as a Good Video Caption?

Read the original on arXiv Computer Vision →

The paper introduces CodeCap-Bench, a benchmark for evaluating whether executable code can serve as an effective video caption. It compares code captions to natural language captions on tasks such as question answering and video reconstruction, finding that code captions yield higher reasoning accuracy and reconstruction fidelity. The study also assesses current multimodal large language models, noting that while proprietary models show promise, they still struggle with complex temporal dynamics, and open‑source models lag significantly.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI

Sep 21

VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration

VidOmni-Bench is a new benchmark for fine‑grained video understanding that asks models to verify whether each event in dense video captions is supported by the video. It contains 500 videos covering five complexity types and durations from 4 seconds to 90 minutes, and uses human‑verified sentence‑level labels to create hard negatives. Experiments show that Video‑LLMs often hallucinate events, struggle to detect incorrect descriptions, and exhibit varying weaknesses depending on video complexity and duration.

By Changbeen Kim, Junwon Chang, Kipyo Kim, Risa Shinoda, Kuniaki Saito, Donghyun Kim

llms nlp benchmarks

arXiv AI

Jun 24

CineCap: Structured Reasoning with Spatio-Temporal Anchors for Cinematographic Video Captioning

arXiv:2606. 24636v1 Announce Type: new Abstract: Cinematographic captioning aims to describe how a video is filmed using professional film-language concepts such as camera movement, shot size, depth of field, composition, and shooting angle.

By Xinyu Mao, Yuhui Zeng, Xiaokun Liu, Wenyu Qin, Meng Wang, Xin Tao, Pengfei Wan, Xiaohan Xing, Max Meng

llms diffusion reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Sep 4

Seeing Before Synthesizing: VLM-Guided Transition Event Discovery for Weakly-Supervised Dense Video Captioning

The paper introduces Seeing Before Synthesizing (SBS), a weakly-supervised dense video captioning framework that uses a vision‑language model to generate frame‑level narratives for gaps between events and detect transitions based on semantic changes. SBS refines temporal masks by aligning transition points with vision‑language cues, rather than relying on rigidly placed synthetic captions. Experiments on ActivityNet Captions and YouCook2 show that SBS achieves state‑of‑the‑art results in both captioning and localization tasks.

By Ye-Chan Kim, Seunghee Choi, SeungJu Cha, Si-Woo Kim, Hwiseon Kim, Hyungee Kim, Dong-Jin Kim

llms multimodal benchmarks safety

arXiv Computer Vision

Sep 10

Putting Captions to the Test: Evaluating Video Caption Quality through Multiple-Choice Question Answering

arXiv:2609.09973v1 Announce Type: new Abstract: Evaluating video captioning remains a critical challenge for Visual Large Language Models (VLLMs). Existing metrics primarily rely on matching generate...

By Zizhen Wang, Bo Feng, Zhengfeng Lai, Shiyu Li, Yang Lu, Meng Cao, Ping Huang, Xiaoming Simon Wang

llms nlp benchmarks

arXiv AI

Jun 2

CaptionFormer: Unified Segmentation, Tracking, and Captioning for Spatio-Temporal Objects

arXiv:2510. 14904v4 Announce Type: replace-cross Abstract: Dense Video Object Captioning (DVOC) is the task of jointly detecting, tracking, and captioning object trajectories in a video, requiring the ability to understand spatio-temporal details and describe them in natural language.

By Gabriel Fiastre, Antoine Yang, Cordelia Schmid

computer-vision multimodal benchmarks

arXiv Machine Learning

Sep 1

SynMulti: Synthetic-to-Real Learning for Multimodal Video Understanding

arXiv:2604.12335v2 Announce Type: replace-cross Abstract: Training multimodal large language models (MLLMs) for video understanding requires large-scale annotated data spanning diverse tasks such as...

By Tanzila Rahman, Renjie Liao, Leonid Sigal

llms computer-vision nlp fine-tuning multimodal

One email a morning, machine-written

Related stories

VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration

CineCap: Structured Reasoning with Spatio-Temporal Anchors for Cinematographic Video Captioning

Seeing Before Synthesizing: VLM-Guided Transition Event Discovery for Weakly-Supervised Dense Video Captioning

Putting Captions to the Test: Evaluating Video Caption Quality through Multiple-Choice Question Answering

CaptionFormer: Unified Segmentation, Tracking, and Captioning for Spatio-Temporal Objects

SynMulti: Synthetic-to-Real Learning for Multimodal Video Understanding