PercepCap is a video captioning framework that explicitly models spatio‑temporal perception before generating captions. It follows a perceive‑describe chain, first producing a perception trace of object trajectories and temporal events, then generating the final caption conditioned on that trace. The method uses a two‑stage training strategy—supervised fine‑tuning followed by perception‑grounded reinforcement learning—and builds caption‑aligned perception data to ensure the perception trace and caption refer to the same objects and events.
Video captioning requires fine-grained spatio-temporal understanding of videos, including spatial perception of where objects are located and temporal perception of when events occur. Existing MLLMs usually generate captions directly from video inputs without exposing the perceptual evidence behind descriptions.
OneStreamer is a streaming video model that jointly learns to record evidence and respond to tasks through a shared proactive generation process. Its Proactive Hierarchical Caption Memory creates time‑grounded local‑detail captions and event summaries, while Proactive State Transition Learning reduces waiting states by supervising all output anchors. The authors also built a large OneStreamer‑1M dataset and show that a 4B model outperforms baselines on eight streaming video benchmarks, with ablations confirming the benefits of generated captions and PSTL.
arXiv:2510. 14904v4 Announce Type: replace-cross Abstract: Dense Video Object Captioning (DVOC) is the task of jointly detecting, tracking, and captioning object trajectories in a video, requiring the ability to understand spatio-temporal details and describe them in natural language.
arXiv:2610.16172v1 Announce Type: cross Abstract: Recent multimodal large language models (MLLMs) achieve strong results on video question answering, yet often fail to recover the temporal structure...
VidOmni-Bench is a new benchmark for fine‑grained video understanding that asks models to verify whether each event in dense video captions is supported by the video. It contains 500 videos covering five complexity types and durations from 4 seconds to 90 minutes, and uses human‑verified sentence‑level labels to create hard negatives. Experiments show that Video‑LLMs often hallucinate events, struggle to detect incorrect descriptions, and exhibit varying weaknesses depending on video complexity and duration.
MVRCap is a new video captioning framework that treats caption generation as a reinforcement learning problem guided by structured verification. It introduces a Fine‑Grained Verifiable Reward system built on three pillars—Hierarchical Concept Coverage, Factuality Verification, and Conciseness Constraint—supported by the newly created MVRCap‑QA and MVRCap‑SFT datasets. Experiments show that MVRCap outperforms existing methods on benchmarks such as VDC, CAPability, and VC4VG.
The paper introduces Seeing Before Synthesizing (SBS), a weakly-supervised dense video captioning framework that uses a vision‑language model to generate frame‑level narratives for gaps between events and detect transitions based on semantic changes. SBS refines temporal masks by aligning transition points with vision‑language cues, rather than relying on rigidly placed synthetic captions. Experiments on ActivityNet Captions and YouCook2 show that SBS achieves state‑of‑the‑art results in both captioning and localization tasks.
arXiv:2606. 29531v1 Announce Type: cross Abstract: We propose MotionAtlas, a system for detailed captioning of motion-centric videos, comprising (1) a dedicated human-annotated benchmark, (2) a scalable, high-quality pipeline to construct training samples, and (3) a family of powerful Video-MLLMs.
Improving video captioning quality typically demands retraining large vision-language models, an expensive and often impractical requirement. Existing training-free alternatives instead ground captions in detected objects to curb hallucination, but apply only a single, fixed correction pass without prioritizing which objects matter most, leaving semantically significant content omitted.
arXiv:2610.37471v1 Announce Type: cross Abstract: Fine-grained visual perception remains a bottleneck during post-training of multimodal large language models (MLLMs), as perceptual errors can constr...
arXiv:2505.01583v2 Announce Type: replace Abstract: Understanding causal event relationships and achieving fine-grained temporal grounding in videos remain challenging for vision-language models (VLM...
The paper introduces OACap, a caption-guided active observation system for detailed video captioning that requests targeted observations to fill missing information and iteratively refines its captions. OACap consistently improves performance on general video-understanding benchmarks while using only a text‑only reader. The authors also present FSCap‑Bench, a new evaluation suite with Core‑Support Factuality Score and Cross‑Modal Semantic Round‑Trip Score to assess factual and semantic preservation in captions.