The paper investigates why long image captions often fail, arguing that failure depends not just on what facts are mentioned but also on where they appear in the caption. It introduces a position‑aware rubric that breaks down dense human references into atomic visual claims, evaluates generated captions against these claims, and identifies two distinct failure modes: unsupported specificity in the middle and vague, non‑evidential language at the end. Building on this diagnosis, the authors propose PAGER, a reinforcement‑learning objective that provides separate rewards for supported middle evidence, evidence‑bearing tail continuation, claim coverage, and length control, leading to improved position‑wise grounding and caption preference on detailed‑caption benchmarks.
arXiv:2609.36893v1 Announce Type: new
Abstract: Detailed image captioning requires accurate and comprehensive descriptions of fine-grained visual content, yet caption quality spans factual accuracy,...
By Zhenwen Ji, Lei Jin, Shanyong Wang, Jiaming Lu, Chengqiang Lu, Yi Wu, Yao Hu, Lizhen Cui, Yanyu Xu
arXiv:2606. 17678v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) integrate strong text reasoning with visual inputs, yet their responses can be inconsistent with the underlying images, indicating ineffective utilization of visual evidence during inference.
By Yilian Liu, Sicong Leng, Guoshun Nan, Junyi Zhu, Jiayu Huang, Minghao Sun, Xuancheng Zhu, Yisong Chen, Zexian Wei, Xiaofeng Tao
arXiv:2605. 01733v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) hallucinate objects that are not present, and a growing line of work tries to curb this by feeding the model its own generated caption as auxiliary evidence -- assuming that a caption, once available, is something to consume.
By Zeshang Li, Shuoyang Zhang
arXiv:2609.00591v1 Announce Type: new
Abstract: An image may be worth a thousand words, but most captioning models describe it in only a few. Modern vision-language models produce fluent high-level c...
By Suryaansh Jain, Rahasya Barkur, Vishal G, Ryan Rossi, Franck Dernoncourt, Jack Wang, Koustava Goswami, Nedim Lipka, Puneet Mathur, Samyadeep Basu, Seunghyun Yoon
LineupRL introduces a reinforcement learning framework with verifiable rewards for time series captioning, using a frozen large language model to identify the correct time series from a set of distractors based on a generated caption. This approach bypasses the limitations of supervised fine‑tuning and traditional RL rewards that poorly transfer to open‑ended time series generation. Experiments on two captioning benchmarks, as well as forecasting and reconstruction tasks, show that LineupRL outperforms both SFT and RL baselines across all metrics, and its trained 3B vision‑language model surpasses a 72B model distilled from SFT captions. The method also demonstrates resistance to reward hacking and produces captions that accurately trace trends and name key values.
By Haochen Zhang, Laura Yao, Zachary Plotkin, Gengwei Zhang, Tianlong Chen
Object hallucination in multimodal large language models arises when language priors and corpus co-occurrence bias outweigh the visual evidence, with nothing tying an individual object mention to what the image shows. Most remedies intervene at decoding time without training, yet under a unified protocol their benefit is confined to short captions;supervised fine-tuning (SFT) on a detail- rich corpus lengthens captions, but over forty percent still name absent objects.
ZoomCap is a method for fine‑grained captioning that treats missing visual evidence as recoverable information. It uses a draft caption to guide conditional knowledge retrieval, OCR‑aware reading, coverage‑gap recovery, and adaptive visual zooming, focusing deeper crops on complex regions and deriving spatial relations from grounded geometry. The accompanying FactCycle framework evaluates caption claims against the source image and a reconstruction, distinguishing fabrication from generator loss, and the approach shows strong performance gains in both general and medical vision‑language pre‑training.
By Hao Shi, Jiafeng Wang, Yuzhuo Fu, Tao Zhang, Jian Bai, Juelu Zhang, Qing Cui, Chao Huang
arXiv:2610.17706v1 Announce Type: cross
Abstract: Large vision-language models often exhibit object hallucination, generating claims that are not supported by the input image. To mitigate this issue,...
By Hyeonggeun Han, Taehyun Cho, Sangwoo Hong
arXiv:2610.16441v1 Announce Type: cross
Abstract: Current image-text evaluation often relies on a global score, which gives limited insight into which semantic details are supported by the image. Exp...
By Subhadeep Roy, Christoph Leiter, Steffen Eger
arXiv:2608.21819v1 Announce Type: cross
Abstract: Reliable image captioning in Vision-Language Models (VLMs) requires captions to be both precise and complete, avoiding unsupported object mentions wh...
By Jihyung Ko, Eunji Jung, Hyeongsub Kim, Ziseok Lee, Jae Won Cho, Sanghyun Jo, Kyungsu Kim
In this paper, we introduce Claim-Level Rubric Rewards (CuRe), a structured reward framework designed to address the reward-design bottleneck in reinforcement learning for dense video captioning. Existing reward designs generally fall into two categories: holistic response-level judgment across heterogeneous criteria, or alignment-based evaluation against reference captions.