The paper introduces OVRSISBench, a unified benchmark for open‑vocabulary remote sensing image segmentation, and evaluates existing OVS/OVRSIS models, uncovering their shortcomings in remote sensing contexts. Leveraging insights from this evaluation, the authors propose RSKT‑Seg, a new framework featuring a Multi‑Directional Cost Map Aggregation module, an Efficient Cost Map Fusion transformer, and a Remote Sensing Knowledge Transfer module. Experiments on the benchmark demonstrate that RSKT‑Seg outperforms strong baselines by +3.8 mIoU and +5.9 mACC while achieving twice the inference speed.
By Bingyu Li, Haocheng Dong, Da Zhang, Zhiyuan Zhao, Junyu Gao, Xuelong Li
Prompt-driven vision-language models (VLMs) hold immense promise for accelerating dense remote sensing (RS) annotation, but static models suffer from severe performance degradation when deployed on novel scenes, unseen categories, or visually confusing backgrounds. Moreover, existing unified paradigms primarily rely on intra-image specific prompts, lacking flexible task routing to adapt to multi-intent operational workflows.
Open-vocabulary remote sensing segmentation has recently emerged as a promising paradigm that enables pixel-level recognition of arbitrary categories specified by natural language, including classes unseen during training. However, geospatial domain shifts caused by heterogeneous regions, spatial resolutions, and acquisition platforms weaken visual-text matching and limit cross-dataset generalization.
The paper introduces Text-to-Seed (T2S), a training‑free framework for open‑vocabulary semantic segmentation that repurposes Stable Diffusion to generate attention‑based seed points from text queries. These sparse seeds serve as point prompts for the Segment Anything Model (SAM), enabling reliable region expansion without relying on inaccurate coarse masks. T2S achieves strong performance on standard OVSS benchmarks using only the text‑to‑region correspondence of diffusion models and no task‑specific training or extra annotations.
By Kumju Jo, Heesun Jung, Sungyong Baik
arXiv:2605.19410v2 Announce Type: replace
Abstract: Segmentation has become easy when the concept is known, requiring retrieval of a learned visual grounding from text. It remains hard for open ad-ho...
By Zilin Wang, Stella X. Yu
PANORAMA introduces a new panoptic grounded captioning framework that jointly generates detailed image captions and associates each phrase with precise pixel-level masks. The authors create PanoCaps, a human‑annotated benchmark with dense captions and near‑complete pixel coverage, and propose a phrase‑mask matching protocol with a generalized Panoptic Quality metric. PANORAMA conditions a pretrained segmenter on contextualized phrase representations, learns to select appropriate masks, and achieves state‑of‑the‑art grounding performance on PanoCaps and other pixel‑level tasks.
By Sara Pieri, Evangelos Kazakos, Shizhe Chen, Josef Sivic, Cordelia Schmid