The paper examines the intermediate‑entropy regime of natural language generation, where a single input can have multiple valid outputs such as in peer‑review generation and image captioning. It introduces a target‑side consolidation method that merges all references into a single target preserving every distinct information unit, and presents the MERC‑67K dataset of 36,537 papers and 31,014 images with both original and consolidated references. Human evaluations show that models trained on consolidated references achieve higher coverage and overall quality scores, outperforming multi‑reference baselines across several large language models while using fewer individual references during training.
arXiv:2609.22805v1 Announce Type: new Abstract: Natural language generation (NLG) tasks span the spectrum of conditional entropy, ranging from highly constrained machine translation to open-ended dia...
The paper presents a matched‑budget audit framework for evaluating recaptioned image‑text supervision distributions, which are created by applying a captioning policy, captioner, and source corpus to generate dense descriptions for text‑to‑image training. The framework operates under a fixed text budget of 64 tokens and produces a five‑axis profile covering prompt coverage, image‑conditioned faithfulness, and caption surface quality, using controllable basic units as a common claim metric. The authors apply the framework to seven paired comparisons across five public corpora, demonstrating improvements in supported claim units and revealing consistent long‑vs‑dense trade‑offs across different judges and budgets, and they release the audited multi‑source recap corpus and audit artifacts.
DEPICT is a new training‑free metric for evaluating text‑to‑image alignment. It replaces fixed reference answers with an agreement rule that compares image‑based and caption‑only responses, weighting questions by how decisively the caption determines them. By merging this agreement score with a holistic score, DEPICT improves negation accuracy dramatically and outperforms existing training‑free metrics while matching or exceeding fine‑tuned evaluators on several benchmarks.
arXiv:2609.36893v1 Announce Type: new Abstract: Detailed image captioning requires accurate and comprehensive descriptions of fine-grained visual content, yet caption quality spans factual accuracy,...
arXiv:2607. 28986v1 Announce Type: cross Abstract: Zero-shot image captioning (ZIC) describes images without paired image-caption supervision during captioner training, relying on text-only corpora and frozen pretrained image-text scorers.
arXiv:2606.29997v2 Announce Type: replace Abstract: Automatic evaluation of image and video captioning is essential for benchmarking multimodal systems, although standard evaluation metrics show limi...
The paper introduces Preserve-and-Compose Training (PACT) for composed image retrieval, a task where a query image is modified by a textual instruction while preserving visual content from a reference image. PACT learns from image–text–text triplets, using target captions for supervision and visual evidence from the source image to maintain relevant details, without requiring target images or gallery updates. The authors also propose Chord scoring, which blends target similarity with source-relative directional agreement in a frozen image space, and demonstrate that this combined approach yields strong retrieval performance across multiple zero-shot CIR benchmarks and various backbones.
Automatic evaluation of image and video captioning is essential for benchmarking multimodal systems, although standard evaluation metrics show limited alignment with human judgments. Recent approaches using large language models (LLMs), commonly referred to as LLM-as-a-Judge, have improved alignment with human judgments but still suffer from a mismatch between large-vocabulary language modeling and evaluation over a small label set.
arXiv:2509. 07295v4 Announce Type: replace-cross Abstract: Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture.
The paper introduces Lens, a training‑free framework that aligns multimodal representations with the semantic perspective required by downstream tasks. Lens uses a task‑specific readout phrase to anchor the perspective and then aggregates token states after the full input, ensuring the extracted representation reflects task‑conditioned evidence integration rather than generic salient content. The method achieves a Precision@1 of 63.9 across 36 MMEB datasets, outperforming the nearest training‑free baseline by 10.2 points.
Flow-CLIP introduces a method to improve compositional image-text matching by refining global image-text alignment with image-conditioned changes in token–patch evidence. It combines Global Compatibility Calibration (GCC) for sentence-level similarity, Image-Conditioned Evidence Refinement (ICER) to incorporate visual patch context, and Cross-Image Margin Ranking (CIMR) to enhance ranking margins when images support correct descriptions. Experiments demonstrate that Flow-CLIP outperforms pretrained CLIP and other global alignment approaches on binding accuracy and reduces reliance on language priors.
The paper introduces Redemption Score (RS), a multi‑modal evaluation framework for image captioning that combines three complementary signals: Mutual Information Divergence for global image‑text alignment, DINO‑based perceptual similarity of cycle‑generated images for visual grounding, and LLM text embeddings for contextual similarity to human references. RS fuses these signals to provide a more holistic assessment, achieving a Kendall‑τ of 58.42 on Flickr8k and outperforming most prior methods. The framework demonstrates consistent performance across Conceptual Captions and MS COCO, offering a robust evaluation that captures both visual accuracy and text quality.