The paper introduces a position‑aware rubric that breaks down long image captions into atomic visual claims, allowing verification of generated text against these claims and revealing distinct failure modes: unsupported specificity in the middle and vague language at the end. Using this diagnosis, the authors propose PAGER, a reinforcement learning objective that provides separate rewards for middle evidence, tail continuation, claim coverage, and length control. Experiments on detailed‑caption benchmarks show that PAGER improves position‑wise grounding and pairwise caption preference while maintaining overall caption quality.
By Yuanze Hu, Zhichao Yang, Junwei Jing, Xin Yang, Wenxuan Liu, Tinghai Zhang, Jia Fu, Hongzhi Zhang, Xiangyu Wu, Ruiming Tang, Tingting Gao, Han Li
arXiv:2606. 17678v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) integrate strong text reasoning with visual inputs, yet their responses can be inconsistent with the underlying images, indicating ineffective utilization of visual evidence during inference.
By Yilian Liu, Sicong Leng, Guoshun Nan, Junyi Zhu, Jiayu Huang, Minghao Sun, Xuancheng Zhu, Yisong Chen, Zexian Wei, Xiaofeng Tao
arXiv:2609.36893v1 Announce Type: new
Abstract: Detailed image captioning requires accurate and comprehensive descriptions of fine-grained visual content, yet caption quality spans factual accuracy,...
By Zhenwen Ji, Lei Jin, Shanyong Wang, Jiaming Lu, Chengqiang Lu, Yi Wu, Yao Hu, Lizhen Cui, Yanyu Xu
arXiv:2609.00591v1 Announce Type: new
Abstract: An image may be worth a thousand words, but most captioning models describe it in only a few. Modern vision-language models produce fluent high-level c...
By Suryaansh Jain, Rahasya Barkur, Vishal G, Ryan Rossi, Franck Dernoncourt, Jack Wang, Koustava Goswami, Nedim Lipka, Puneet Mathur, Samyadeep Basu, Seunghyun Yoon
arXiv:2605. 01733v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) hallucinate objects that are not present, and a growing line of work tries to curb this by feeding the model its own generated caption as auxiliary evidence -- assuming that a caption, once available, is something to consume.
By Zeshang Li, Shuoyang Zhang
Object hallucination in multimodal large language models arises when language priors and corpus co-occurrence bias outweigh the visual evidence, with nothing tying an individual object mention to what the image shows. Most remedies intervene at decoding time without training, yet under a unified protocol their benefit is confined to short captions;supervised fine-tuning (SFT) on a detail- rich corpus lengthens captions, but over forty percent still name absent objects.
arXiv:2610.17706v1 Announce Type: cross
Abstract: Large vision-language models often exhibit object hallucination, generating claims that are not supported by the input image. To mitigate this issue,...
By Hyeonggeun Han, Taehyun Cho, Sangwoo Hong
ZoomCap is a method for fine‑grained captioning that treats missing visual evidence as recoverable information. It uses a draft caption to guide conditional knowledge retrieval, OCR‑aware reading, coverage‑gap recovery, and adaptive visual zooming, focusing deeper crops on complex regions and deriving spatial relations from grounded geometry. The accompanying FactCycle framework evaluates caption claims against the source image and a reconstruction, distinguishing fabrication from generator loss, and the approach shows strong performance gains in both general and medical vision‑language pre‑training.
By Hao Shi, Jiafeng Wang, Yuzhuo Fu, Tao Zhang, Jian Bai, Juelu Zhang, Qing Cui, Chao Huang
Re$^3$Cap introduces a retrieval‑guided refinement strategy for image captioning that leverages multi‑modal retrieval as a reasoning signal. The method, built on a Caption Refinement Suggester and a Caption Quality Assessor, detects hallucinations and omissions to produce more accurate and detailed captions without extra annotations. Experiments show it surpasses supervised fine‑tuning and improves relation reasoning by 8.64% on the COCO‑LN500 benchmark.
By Haonan Jia, Shichao Dong, Zenghui Sun, Jiawen Zheng, Ziqi Miao, Gege Shi, Qiuyu Zhao, Jinsong Lan, Xiaoyong Zhu, Bo Zheng
In this paper, we introduce Claim-Level Rubric Rewards (CuRe), a structured reward framework designed to address the reward-design bottleneck in reinforcement learning for dense video captioning. Existing reward designs generally fall into two categories: holistic response-level judgment across heterogeneous criteria, or alignment-based evaluation against reference captions.
Generalized CycleGRPO is a unified reinforcement learning framework that jointly trains region understanding and localization across spatial, temporal, and spatio‑temporal domains. It uses a multimodal large language model to generate candidate descriptions for a given region, then localizes each description to reconstruct the original region, providing verifiable rewards for both captioning and localization. The approach extends beyond image masks and boxes to video intervals and timestamped spatial evidence, and introduces the DOVE benchmark to evaluate temporal distinctiveness, achieving significant performance gains on several grounding and temporal tasks.
By Xin Zhang, Haochen Wang, Beibei Lin, Zhaoxiang Zhang, Robby T. Tan, Jason Li, Dacheng Tao
LineupRL introduces a reinforcement learning framework with verifiable rewards for time series captioning, using a frozen large language model to identify the correct time series from a set of distractors based on a generated caption. This approach bypasses the limitations of supervised fine‑tuning and traditional RL rewards that poorly transfer to open‑ended time series generation. Experiments on two captioning benchmarks, as well as forecasting and reconstruction tasks, show that LineupRL outperforms both SFT and RL baselines across all metrics, and its trained 3B vision‑language model surpasses a 72B model distilled from SFT captions. The method also demonstrates resistance to reward hacking and produces captions that accurately trace trends and name key values.
By Haochen Zhang, Laura Yao, Zachary Plotkin, Gengwei Zhang, Tianlong Chen