arXiv:2509.22650v3 Announce Type: replace
Abstract: Most existing approaches to referring segmentation achieve strong performance only through fine-tuning or by composing multiple pre-trained models,...
By Anna Kukleva, Enis Simsar, Alessio Tonioni, Muhammad Ferjad Naeem, Federico Tombari, Jan Eric Lenssen, Bernt Schiele
PANORAMA introduces a new panoptic grounded captioning framework that jointly generates detailed image captions and associates each phrase with precise pixel-level masks. The authors create PanoCaps, a human‑annotated benchmark with dense captions and near‑complete pixel coverage, and propose a phrase‑mask matching protocol with a generalized Panoptic Quality metric. PANORAMA conditions a pretrained segmenter on contextualized phrase representations, learns to select appropriate masks, and achieves state‑of‑the‑art grounding performance on PanoCaps and other pixel‑level tasks.
By Sara Pieri, Evangelos Kazakos, Shizhe Chen, Josef Sivic, Cordelia Schmid
SetCon reformulates open-ended referring segmentation as set-level concept prediction, using LVLM-generated natural-language concepts instead of segmentation-specific tokens. It employs a hierarchical semantic decomposition that first predicts a shared set-level concept and then refines it into fine-grained concept groups aligned with target subsets. The method, supported by a two-stage annotation pipeline adding hierarchical semantic supervision, achieves state-of-the-art results on multiple image and video benchmarks, with performance gains increasing as the number of referred targets grows.
By Zhixiong Zhang, Yizhuo Li, Shuangrui Ding, Yuhang Zang, Shengyuan Ding, Long Xing, Yibin Wang, Qiaosheng Zhang, Jiaqi Wang
PR-MaGIC is a training‑free, test‑time framework that refines prompts for in‑context image segmentation by using gradient flow from the mask decoder of the Segment Anything Model (SAM). It integrates seamlessly into existing in‑context segmentation pipelines and employs a simple top‑1 selection strategy to stabilize performance. Extensive evaluations show that PR‑MaGIC consistently improves segmentation quality across multiple benchmarks without requiring additional training or architectural changes.
By Minjae Lee, Sungwoo Hur, Soojin Hwang, Won Hwa Kim
Intelligent systems that act in the world require image understanding that is both comprehensive and spatially grounded. Current vision-language models (VLMs) can generate fluent and detailed image ca...
The paper introduces Text-to-Seed (T2S), a training‑free framework for open‑vocabulary semantic segmentation that repurposes Stable Diffusion to generate attention‑based seed points from text queries. These sparse seeds serve as point prompts for the Segment Anything Model (SAM), enabling reliable region expansion without relying on inaccurate coarse masks. T2S achieves strong performance on standard OVSS benchmarks using only the text‑to‑region correspondence of diffusion models and no task‑specific training or extra annotations.
By Kumju Jo, Heesun Jung, Sungyong Baik