SVG-Score introduces a human‑aligned evaluation framework for text‑to‑SVG generation, addressing the shortcomings of existing image‑based metrics like CLIPScore that poorly capture SVG‑specific errors such as color, count, and spatial inaccuracies. The authors first demonstrate that CLIP‑based scores are largely insensitive to these errors and that generic Vision‑Language Models respond inconsistently across error types and styles. They then present a human‑annotated Semantic Alignment dataset and develop two complementary evaluators: a CLIP‑based scorer adapted to vector graphics and a VLM judge refined through supervised fine‑tuning and reward‑shaped reinforcement learning, enabling both fast large‑scale and expressive, interpretable assessment of SVG generators.
By Marco Cipriano, Leonardo Zini, Alexandra Schild, Valentin Teutschbein, Afsana Mimi, Marcella Cornia, Lorenzo Baraldi, Gerard de Melo
arXiv:2609.00591v1 Announce Type: new
Abstract: An image may be worth a thousand words, but most captioning models describe it in only a few. Modern vision-language models produce fluent high-level c...
By Suryaansh Jain, Rahasya Barkur, Vishal G, Ryan Rossi, Franck Dernoncourt, Jack Wang, Koustava Goswami, Nedim Lipka, Puneet Mathur, Samyadeep Basu, Seunghyun Yoon
PANORAMA introduces a new panoptic grounded captioning framework that jointly generates detailed image captions and associates each phrase with precise pixel-level masks. The authors create PanoCaps, a human‑annotated benchmark with dense captions and near‑complete pixel coverage, and propose a phrase‑mask matching protocol with a generalized Panoptic Quality metric. PANORAMA conditions a pretrained segmenter on contextualized phrase representations, learns to select appropriate masks, and achieves state‑of‑the‑art grounding performance on PanoCaps and other pixel‑level tasks.
By Sara Pieri, Evangelos Kazakos, Shizhe Chen, Josef Sivic, Cordelia Schmid
Intelligent systems that act in the world require image understanding that is both comprehensive and spatially grounded. Current vision-language models (VLMs) can generate fluent and detailed image ca...
arXiv:2605. 23500v2 Announce Type: replace-cross Abstract: Segmentation is a fundamental task in computer vision, underpinning pixel-level scene understanding and serving as a cornerstone for applications ranging from autonomous perception to medical image analysis.
By Mario Markov (INSAIT, Sofia University "St. Kliment Ohridski"), Stefan Maria Ailuro (INSAIT, Sofia University "St. Kliment Ohridski"), Mohammad Mahdi (INSAIT, Sofia University "St. Kliment Ohridski"), Luc Van Gool (INSAIT, Sofia University "St. Kliment Ohridski"), Danda Pani Paudel (INSAIT, Sofia University "St. Kliment Ohridski")
The paper introduces PM4Bench, a multimodal, multilingual, multi-task benchmark built on a strictly parallel 10‑language corpus, allowing fair cross‑lingual comparison of Large Vision‑Language Models (LVLMs). It also proposes a vision setting that embeds textual inputs directly into images to better mimic real deployment scenarios. Experiments show OCR performance drives cross‑lingual gaps, leading to an OCR‑centric GRPO training strategy that improves multilingual VQA and reduces disparities without costly supervision.
By Junyuan Gao, Jiahe Song, Jiang Wu, Runchuan Zhu, Guanlin Shen, Shasha Wang, Xingjian Wei, Haote Yang, Weijia Li, Bin Wang, Lijun Wu, Conghui He
The study investigates how unified vision‑language models (VLMs) can simultaneously support visual understanding and generation. Using controlled benchmarks (SmartWatch and modified CelebA) that pair VQA, captioning, and text‑to‑image tasks, the authors evaluate several LLM‑based architectures built on SigLIP and VQ‑VAE visual spaces. Results show that mixed training can improve both understanding and generation, but the gains depend on how well the visual input and output spaces are aligned; misaligned or distorted visual spaces can weaken or reverse these benefits. The paper also demonstrates that balancing data across tasks and controlling attribute frequencies can help recover underrepresented visual concepts, and that the transfer is driven more by the base language model’s learned relationships than by visual adapters.
By Jihai Zhang, Tianle Li, Linjie Li, Zhengyuan Yang, Yu Cheng
arXiv:2609.00709v1 Announce Type: cross
Abstract: Large Vision-Language Models produce fluent image descriptions but offer limited semantic control: users cannot reliably specify whether captions sho...
By Jongyeop Hyun, Taeyoung Kim, Hyounghun Kim
The paper introduces Auto-Comp, a fully automated, concept-driven pipeline that generates photorealistic compositional benchmarks for vision‑language models. Auto‑Comp creates paired Minimal and Contextual samples for each concept, enabling isolation of core binding abilities from visio‑linguistic complexity. Evaluations across 25 models reveal consistent failures in attribute and relational binding, with context helping relational tasks but hindering attribute tasks due to visual clutter.
By Cristian Sbrolli, Toshihiko Yamasaki, Matteo Matteucci
The paper presents CycleGRPO, a reinforcement learning framework that unifies region understanding and localization for multimodal large language models (MLLMs). By treating the MLLM as both actor and critic, the method generates region captions and immediately grounds them back into spatial coordinates, using a token‑level cycle‑consistency reward that obviates the need for textual ground truths. Experiments on SAMTok demonstrate that CycleGRPO can bootstrap region captioning, VQA, grounded dialogue, and referring segmentation simultaneously, achieving consistent performance gains without task‑specific fine‑tuning.
By Xin Zhang, Haochen Wang, Yikang Zhou, Zhuochen Wang, Xiangtai Li, Robby T. Tan
Falcon Perception-HD applies reinforcement learning (GRPO) to autoregressive perception models, aligning them directly with precision and recall metrics rather than relying on maximum‑likelihood fine‑tuning. The RL framework introduces reward design for set‑structured outputs and multi‑head sampling control, enabling state‑of‑the‑art performance in very dense scenes (up to 500 objects) and eliminating common issues such as mask repetitions, NMS, and coordinate deduplication. Hybrid self‑annotation pipelines tailored for difficult referring expressions and dense scenes further boost RL training, with improvements observed across all difficulty levels on PBench and SACO‑Gold, and the model preserves object existence knowledge without negative samples.
SVG-Score introduces a human‑aligned evaluation framework for text‑to‑SVG generation, addressing the inadequacies of existing image‑based metrics like CLIPScore that poorly capture SVG‑specific errors such as color, count, and spatial inaccuracies. The authors first demonstrate that CLIP‑based scores are largely insensitive to these errors and that off‑the‑shelf Vision‑Language Models respond unevenly across error types and styles. They then create a human‑annotated Semantic Alignment dataset and develop two evaluators: a CLIP scorer adapted to vector graphics and a VLM judge trained with supervised fine‑tuning and reinforcement learning, enabling both fast large‑scale and expressive, interpretable assessment of SVG generators.