arXiv Computer Vision
Beyond Anonymous Captions: Grounding Character Identity in Video Captioning and Question Answering
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Sep 21
VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration
VidOmni-Bench is a new benchmark for fine‑grained video understanding that asks models to verify whether each event in dense video captions is supported by the video. It contains 500 videos covering five complexity types and durations from 4 seconds to 90 minutes, and uses human‑verified sentence‑level labels to create hard negatives. Experiments show that Video‑LLMs often hallucinate events, struggle to detect incorrect descriptions, and exhibit varying weaknesses depending on video complexity and duration.
arXiv AI
Jul 14
Evidence-Backed Video Question Answering
arXiv:2607. 11862v1 Announce Type: cross Abstract: Current Video Large Language Models (Video LLMs) excel in question answering (QA) but largely operate as black boxes, providing textual answers without verifiable visual grounding.
arXiv Computer Vision
Oct 5
DEPICT: Scoring Text-to-Image Alignment by Answer Agreement
DEPICT is a new training‑free metric for evaluating text‑to‑image alignment. It replaces fixed reference answers with an agreement rule that compares image‑based and caption‑only responses, weighting questions by how decisively the caption determines them. By merging this agreement score with a holistic score, DEPICT improves negation accuracy dramatically and outperforms existing training‑free metrics while matching or exceeding fine‑tuned evaluators on several benchmarks.
arXiv AI
Sep 17
CapMem: A Benchmark for Caption-Based Episodic Memory in Egocentric Video
CapMem is a new benchmark for evaluating caption-based episodic memory in egocentric video. It contains 75 videos (33.7 hours total) and 1,000 multiple-choice questions across 16 scenarios, designed to test the Episodic Memory Video Caption QA task. Experiments show that using captions as memory outperforms direct VideoQA on long videos, and a caption-guided retrieve-and-verify approach further boosts accuracy.
arXiv Computer Vision
Sep 10
Putting Captions to the Test: Evaluating Video Caption Quality through Multiple-Choice Question Answering
arXiv:2609.09973v1 Announce Type: new Abstract: Evaluating video captioning remains a critical challenge for Visual Large Language Models (VLLMs). Existing metrics primarily rely on matching generate...
arXiv Computer Vision
3d ago
Grounded Human-Attributed Description and Activity Recognition in Videos
The paper introduces Grounded Human-Attributed Description and Activity Recognition (GHADAR), a task that requires describing the characteristics and activities of every person in a video given their locations. It presents a new dataset derived from AVA-Actions with open‑set captions, a method that constrains cross‑attention masks in vision‑language models to better use grounding information, and a holistic VLM‑based evaluation schema that compares concepts directly between video and predictions. Experiments show the proposed method surpasses state‑of‑the‑art VLMs and highlight limitations of existing evaluation metrics.