Hugging Face Trending Papers

Where Do Long Captions Fail? Position-Aware Diagnosis and Reinforcement Learning for Detailed Image Captioning

Read the original on Hugging Face Trending Papers →

The paper investigates why long image captions often fail, arguing that failure depends not just on what facts are mentioned but also on where they appear in the caption. It introduces a position‑aware rubric that breaks down dense human references into atomic visual claims, evaluates generated captions against these claims, and identifies two distinct failure modes: unsupported specificity in the middle and vague, non‑evidential language at the end. Building on this diagnosis, the authors propose PAGER, a reinforcement‑learning objective that provides separate rewards for supported middle evidence, evidence‑bearing tail continuation, claim coverage, and length control, leading to improved position‑wise grounding and caption preference on detailed‑caption benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv Computer Vision
Oct 23

Where Do Long Captions Fail? Position-Aware Diagnosis and Reinforcement Learning for Detailed Image Captioning

The paper introduces a position‑aware rubric that breaks down long image captions into atomic visual claims, allowing verification of generated text against these claims and revealing distinct failure modes: unsupported specificity in the middle and vague language at the end. Using this diagnosis, the authors propose PAGER, a reinforcement learning objective that provides separate rewards for middle evidence, tail continuation, claim coverage, and length control. Experiments on detailed‑caption benchmarks show that PAGER improves position‑wise grounding and pairwise caption preference while maintaining overall caption quality.

By Yuanze Hu, Zhichao Yang, Junwei Jing, Xin Yang, Wenxuan Liu, Tinghai Zhang, Jia Fu, Hongzhi Zhang, Xiangyu Wu, Ruiming Tang, Tingting Gao, Han Li
arXiv AI
Jun 17

See First, Answer Later: Visual Evidence Pre-Alignment via Sufficiency-Driven RL

arXiv:2606. 17678v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) integrate strong text reasoning with visual inputs, yet their responses can be inconsistent with the underlying images, indicating ineffective utilization of visual evidence during inference.

By Yilian Liu, Sicong Leng, Guoshun Nan, Junyi Zhu, Jiayu Huang, Minghao Sun, Xuancheng Zhu, Yisong Chen, Zexian Wei, Xiaofeng Tao
arXiv Computer Vision
Sep 2

A Glance Is All You Need: Single-Pass Fine-Grained Image Captioning with SimLoss

arXiv:2609.00591v1 Announce Type: new Abstract: An image may be worth a thousand words, but most captioning models describe it in only a few. Modern vision-language models produce fluent high-level c...

By Suryaansh Jain, Rahasya Barkur, Vishal G, Ryan Rossi, Franck Dernoncourt, Jack Wang, Koustava Goswami, Nedim Lipka, Puneet Mathur, Samyadeep Basu, Seunghyun Yoon
Hugging Face Trending Papers
Aug 13

Dual-Stream Cross-Anchor Correction Grounding Long-Form Captions and the Domain Limits of Object-Level Anchors

Object hallucination in multimodal large language models arises when language priors and corpus co-occurrence bias outweigh the visual evidence, with nothing tying an individual object mention to what the image shows. Most remedies intervene at decoding time without training, yet under a unified protocol their benefit is confined to short captions;supervised fine-tuning (SFT) on a detail- rich corpus lengthens captions, but over forty percent still name absent objects.