The paper introduces Seeing Before Synthesizing (SBS), a weakly-supervised dense video captioning framework that uses a vision‑language model to generate frame‑level narratives for gaps between events and detect transitions based on semantic changes. SBS refines temporal masks by aligning transition points with vision‑language cues, rather than relying on rigidly placed synthetic captions. Experiments on ActivityNet Captions and YouCook2 show that SBS achieves state‑of‑the‑art results in both captioning and localization tasks.
arXiv:2510. 14904v4 Announce Type: replace-cross Abstract: Dense Video Object Captioning (DVOC) is the task of jointly detecting, tracking, and captioning object trajectories in a video, requiring the ability to understand spatio-temporal details and describe them in natural language.
MEV is a multi‑event video dataset created to support long‑take video generation. It contains high‑quality real‑world videos where subjects perform distinct, non‑overlapping events with precise temporal boundaries and fine‑grained procedural annotations. The dataset includes a comprehensive annotation pipeline that segments videos into temporally grounded events and generates structured captions at both global and event levels, enabling event‑centric evaluation and improved long‑take event transitions in generation experiments.
VidOmni-Bench is a new benchmark for fine‑grained video understanding that asks models to verify whether each event in dense video captions is supported by the video. It contains 500 videos covering five complexity types and durations from 4 seconds to 90 minutes, and uses human‑verified sentence‑level labels to create hard negatives. Experiments show that Video‑LLMs often hallucinate events, struggle to detect incorrect descriptions, and exhibit varying weaknesses depending on video complexity and duration.
arXiv:2505.01583v2 Announce Type: replace Abstract: Understanding causal event relationships and achieving fine-grained temporal grounding in videos remain challenging for vision-language models (VLM...
PercepCap is a video captioning framework that explicitly models spatio‑temporal perception before generating captions. It follows a perceive‑describe chain, first producing a perception trace of object trajectories and temporal events, then generating the final caption conditioned on that trace. The method uses a two‑stage training strategy—supervised fine‑tuning followed by perception‑grounded reinforcement learning—and builds caption‑aligned perception data to ensure the perception trace and caption refer to the same objects and events.
Video captioning requires fine-grained spatio-temporal understanding of videos, including spatial perception of where objects are located and temporal perception of when events occur. Existing MLLMs usually generate captions directly from video inputs without exposing the perceptual evidence behind descriptions.
The paper introduces CodeCap-Bench, a benchmark for evaluating whether executable code can serve as an effective video caption. It compares code captions to natural language captions on tasks such as question answering and video reconstruction, finding that code captions yield higher reasoning accuracy and reconstruction fidelity. The study also assesses current multimodal large language models, noting that while proprietary models show promise, they still struggle with complex temporal dynamics, and open‑source models lag significantly.
EventVL introduces the first generative event-based multimodal large language model (MLLM) designed for explicit semantic understanding of event streams. The framework leverages a newly annotated dataset of nearly 1.4 million event–image/video–text pairs and incorporates an Event Spatiotemporal Representation to capture comprehensive event information, along with Dynamic Semantic Alignment to refine sparse semantic spaces. Experiments demonstrate that EventVL outperforms existing MLLM baselines in event captioning and scene description generation tasks, advancing the field of event vision.
arXiv:2607. 02963v1 Announce Type: cross Abstract: Dense video captioning aims to generate temporally grounded descriptions of video events, benefiting both event-level video understanding and generation.
EVA-Cap is a new framework for audiovisual video captioning that breaks captions into atomic audiovisual events, turning unstructured text supervision into event‑centric alignment. The model is trained with supervised fine‑tuning and Group Relative Policy Optimization on a self‑curated dataset, EVA‑Data, which applies event‑centric quality control. Experiments show EVA‑Cap outperforms prior methods on event‑centric metrics (e.g., +9.0% on ChronusAV) while remaining competitive on general captioning benchmarks (e.g., +3.3% on WorldSense).
arXiv:2601.12243v3 Announce Type: replace-cross Abstract: Video summarization transforms long videos into concise representations that are easier to document and analyze, especially in high-stakes do...
arXiv:2606. 00987v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have shown strong visual understanding and language-guided grounding abilities, yet their capacity for multi-temporal visual reasoning remains underexplored.