The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
VidOmni-Bench is a new benchmark for fine‑grained video understanding that asks models to verify whether each event in dense video captions is supported by the video. It contains 500 videos covering five complexity types and durations from 4 seconds to 90 minutes, and uses human‑verified sentence‑level labels to create hard negatives. Experiments show that Video‑LLMs often hallucinate events, struggle to detect incorrect descriptions, and exhibit varying weaknesses depending on video complexity and duration.
arXiv:2609.09973v1 Announce Type: new Abstract: Evaluating video captioning remains a critical challenge for Visual Large Language Models (VLLMs). Existing metrics primarily rely on matching generate...
Automatic evaluation of image and video captioning is essential for benchmarking multimodal systems, although standard evaluation metrics show limited alignment with human judgments. Recent approaches using large language models (LLMs), commonly referred to as LLM-as-a-Judge, have improved alignment with human judgments but still suffer from a mismatch between large-vocabulary language modeling and evaluation over a small label set.
arXiv:2606.29997v2 Announce Type: replace Abstract: Automatic evaluation of image and video captioning is essential for benchmarking multimodal systems, although standard evaluation metrics show limi...
The paper presents a batik-captioning system that uses class labels as compact supervision when expert text annotations are scarce. It combines a LoRA‑adapted DINOv2 classifier with a Qwen2.5‑VL‑7B captioner, both loaded in 4‑bit NF4, and evaluates three captioner conditions across 150 images, achieving a 0.9067 ground‑truth class‑label mention rate. The classifier attains 96.10% pooled accuracy, with higher performance on public‑source images (98.33%) than on local images (75.76%), and the study highlights how class conditioning preserves label information while also propagating classification errors into fluent captions.
arXiv:2609.36628v1 Announce Type: new Abstract: Vision-Language Models (VLMs) can generate rich video captions, yet often misidentify which person performs an action or which limb is involved, partic...