arXiv Machine Learning September 2, 2026 By Jongyeop Hyun, Taeyoung Kim, Hyounghun Kim

Controllable Image Captioning with Prompt-Conditioned Scene Rewards

Read the original on arXiv Machine Learning →

The Flow has not summarised this story yet — read it at arXiv Machine Learning.

arXiv Computer Vision

2d ago

A Glance Is All You Need: Single-Pass Fine-Grained Image Captioning with SimLoss

arXiv:2609.00591v1 Announce Type: new Abstract: An image may be worth a thousand words, but most captioning models describe it in only a few. Modern vision-language models produce fluent high-level c...

By Suryaansh Jain, Rahasya Barkur, Vishal G, Ryan Rossi, Franck Dernoncourt, Jack Wang, Koustava Goswami, Nedim Lipka, Puneet Mathur, Samyadeep Basu, Seunghyun Yoon

llms rag fine-tuning efficiency multimodal

Hugging Face Trending Papers

Jul 23

ProCap: Prominence-guided Object Rectification for Faithful and Comprehensive Video Captioning

Improving video captioning quality typically demands retraining large vision-language models, an expensive and often impractical requirement. Existing training-free alternatives instead ground captions in detected objects to curb hallucination, but apply only a single, fixed correction pass without prioritizing which objects matter most, leaving semantically significant content omitted.

llms multimodal safety

arXiv AI

Aug 24

Re$^3$Cap: Retrieval-Guided Refinement for Image Captioning Enhancement via Reinforcement Learning

Re$^3$Cap introduces a retrieval‑guided refinement strategy for image captioning that leverages multi‑modal retrieval as a reasoning signal. The method, built on a Caption Refinement Suggester and a Caption Quality Assessor, detects hallucinations and omissions to produce more accurate and detailed captions without extra annotations. Experiments show it surpasses supervised fine‑tuning and improves relation reasoning by 8.64% on the COCO‑LN500 benchmark.

By Haonan Jia, Shichao Dong, Zenghui Sun, Jiawen Zheng, Ziqi Miao, Gege Shi, Qiuyu Zhao, Jinsong Lan, Xiaoyong Zhu, Bo Zheng

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Jun 10

SD-GRPO: Verifiable Segment Decomposition for Long-Form Vision-Language Generation

arXiv:2606. 09871v1 Announce Type: cross Abstract: Group Relative Policy Optimization (GRPO) and its variants, originally developed for Large Language Models (LLMs), have recently been applied to Multimodal LLMs and produced strong results.

By Hyunwoong Kim, Seongeun Lee, Hannah Yun, Junhyun Park, Jonggwon Park

llms computer-vision multimodal

arXiv Computer Vision

4d ago

Focus Where It Counts: A Salience-Driven Vision-Language Model for Low Vision Assistance

The paper introduces Salience-LLaVA, a vision‑language model that prioritizes scene elements based on their importance for low‑vision users. It presents three new salience‑aware datasets—Salience COCO, Salience Flickr, and Salience VizWiz—annotated with object‑level salience verified by low‑vision participants. The authors also propose the SCMI metric to evaluate caption ordering accuracy and demonstrate the system’s practicality by deploying it on assistive glasses.

By Jiazhao Liang, Hao Huang, Shuaihang Yuan, Congcong Wen, Geeta Chandra Raju Bethala, Giles Hamilton-Fletcher, Yu Hao, John-Ross Rizzo, Mengyu Wang, Anthony Tzes, Yi Fang

llms multimodal

Hugging Face Trending Papers

Jul 6

Claim-Level Rubric Rewards for Video Caption Reinforcement Learning

In this paper, we introduce Claim-Level Rubric Rewards (CuRe), a structured reward framework designed to address the reward-design bottleneck in reinforcement learning for dense video captioning. Existing reward designs generally fall into two categories: holistic response-level judgment across heterogeneous criteria, or alignment-based evaluation against reference captions.

reinforcement-learning safety