arXiv Computation and Language October 16, 2026 By Maitreya Prafulla Chitale, Ketaki Mangesh Shetye, Yash More, Harshit Gupta, Manav Chaudhary, Manish Shrivastava, Vasudeva Varma

One Reference or Many? Rethinking Multi-Reference Training for Intermediate-Entropy Text Generation

Read the original on arXiv Computation and Language →

The paper examines the intermediate‑entropy regime of natural language generation, where a single input can have multiple valid outputs such as in peer‑review generation and image captioning. It introduces a target‑side consolidation method that merges all references into a single target preserving every distinct information unit, and presents the MERC‑67K dataset of 36,537 papers and 31,014 images with both original and consolidated references. Human evaluations show that models trained on consolidated references achieve higher coverage and overall quality scores, outperforming multi‑reference baselines across several large language models while using fewer individual references during training.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language

Sep 22

To Consolidate or not to Consolidate? Evaluating the Impact of Consolidation in Multi-Reference Training using Peer Reviews

arXiv:2609.22805v1 Announce Type: new Abstract: Natural language generation (NLG) tasks span the spectrum of conditional entropy, ranging from highly constrained machine translation to open-ended dia...

By Maitreya Prafulla Chitale, Ketaki Mangesh Shetye, Yash More, Harshit Gupta, Manav Chaudhary, Manish Shrivastava, Vasudeva Varma

nlp benchmarks

arXiv AI

Oct 2

A Matched-Budget Audit Framework for Recaptioned Image-Text Supervision Distributions

The paper presents a matched‑budget audit framework for evaluating recaptioned image‑text supervision distributions, which are created by applying a captioning policy, captioner, and source corpus to generate dense descriptions for text‑to‑image training. The framework operates under a fixed text budget of 64 tokens and produces a five‑axis profile covering prompt coverage, image‑conditioned faithfulness, and caption surface quality, using controllable basic units as a common claim metric. The authors apply the framework to seven paired comparisons across five public corpora, demonstrating improvements in supported claim units and revealing consistent long‑vs‑dense trade‑offs across different judges and budgets, and they release the audited multi‑source recap corpus and audit artifacts.

By Giyeong Oh, Junghun Park, Yuhan Bae, Youngjae Yu

llms diffusion multimodal benchmarks

arXiv Computer Vision

Oct 5

DEPICT: Scoring Text-to-Image Alignment by Answer Agreement

DEPICT is a new training‑free metric for evaluating text‑to‑image alignment. It replaces fixed reference answers with an agreement rule that compares image‑based and caption‑only responses, weighting questions by how decisively the caption determines them. By merging this agreement score with a holistic score, DEPICT improves negation accuracy dramatically and outperforms existing training‑free metrics while matching or exceeding fine‑tuned evaluators on several benchmarks.

By Vasco Ramos, Sandra Godinho Silva, Joao Magalhaes, Ricardo Rei, Pedro Henrique Martins

llms diffusion computer-vision fine-tuning multimodal benchmarks safety

arXiv Computation and Language

Sep 30

Momentum-Coupled Rubric Adaptation for Detailed Image Captioning

arXiv:2609.36893v1 Announce Type: new Abstract: Detailed image captioning requires accurate and comprehensive descriptions of fine-grained visual content, yet caption quality spans factual accuracy,...

By Zhenwen Ji, Lei Jin, Shanyong Wang, Jiaming Lu, Chengqiang Lu, Yi Wu, Yao Hu, Lizhen Cui, Yanyu Xu

llms nlp reinforcement-learning fine-tuning benchmarks

arXiv AI

Aug 3

Adjudicated Captioning: Multi-Agent Alignment Scoring and Consensus-Distilled Beam Arbitration for Strict Zero-Shot Image Captioning

arXiv:2607. 28986v1 Announce Type: cross Abstract: Zero-shot image captioning (ZIC) describes images without paired image-caption supervision during captioner training, relying on text-only corpora and frozen pretrained image-text scorers.

By Duy Tran Thanh, Thien-Phuc Doan, Long Nguyen-Vu, Ngo Tan Vu Khanh

llms rag agents efficiency multimodal safety

arXiv Computer Vision

Sep 15

Rigel: Self-Distilled Score Adaptation for Image and Video Captioning Evaluation

arXiv:2606.29997v2 Announce Type: replace Abstract: Automatic evaluation of image and video captioning is essential for benchmarking multimodal systems, although standard evaluation metrics show limi...

By Shuitsu Koyama, Kazuki Matsuda, Yuiga Wada, Shinnosuke Hirano, Daichi Yashima, Komei Sugiura

llms multimodal benchmarks safety