arXiv Computer Vision

DS-BiCA-Net: Pair-Preserving Cross-Temporal Modeling for Remote Sensing Change Captioning

arXiv Computer Vision
Sep 24

From Change Captions to Change Detection: Semantic-Appearance Agreement Framework for Remote Sensing Change Detection

The paper proposes a weakly supervised remote sensing change detection method that uses change captions as the sole supervision signal, eliminating the need for pixel‑level change masks. It introduces a caption‑driven generation pipeline to create bi‑temporal image pairs with controlled changes and a Semantic‑Appearance Agreement Framework (SAAF) that fuses caption‑grounded semantic responses with RGB differences for accurate change localization. Experiments on the Flair‑RSGen and WHU‑CDC datasets demonstrate that SAAF outperforms existing limited‑supervision baselines in macro‑averaged IoU and F1 metrics.

By Yuan Qian, Jie Ma
arXiv Machine Learning
Sep 23

MGRL-RSCC: Multi-Granularity Reward Reinforcement Learning for Fine-Grained Remote Sensing Change Captioning

The paper introduces MGRL-RSCC, a multi‑granularity reward reinforcement learning framework for Remote Sensing Change Captioning (RSCC). It uses a CNN with hierarchical self‑attention to extract visual features, a Transformer decoder for visual‑to‑linguistic translation, and a dual‑decoding strategy combined with token‑level supervised learning and self‑critical reinforcement learning. Three reward functions—linguistic fluency, change state consistency, and structural‑semantic relevance—are employed to improve caption quality and reduce exposure bias and conservative generation.

By Futian Wang, Mengqi Wang, Xiao Wang, Wentao Wu, Haowen Wang, Zhicheng Zhao, Jin Tang
arXiv AI
Oct 27

CoDeRNet: Selective Cross-Task Routing under Heterogeneous Supervision for Change Detection and Captioning

CoDeRNet is a unified framework for Change Detection & Captioning (CDC) that treats the problem as selective information routing between heterogeneous task branches. It introduces a Confidence‑Decomposed Routing (CoDeR) mechanism that decides when and where to transfer information based on receiver needs and sender reliability. Experiments on LEVIR‑MCI and WHU‑CDC demonstrate that CoDeRNet improves caption generation while maintaining strong change localization, showing that conditional, selective cross‑task communication is key to effective CDC under heterogeneous supervision.

By Eunki Cho, Hyeon Bae Kim, Seong Tae Kim
arXiv Computer Vision
4d ago

CompoSET: A Controlled Single-Edit Benchmark for Vision-Language Compositionality

CompoSET is a new vision‑language benchmark that tests compositionality by presenting each image‑caption pair with a foil that differs by only a single controlled edit—such as color, count, spatial relation, or object state—while keeping the overall scene largely unchanged. The dataset contains 1,776 instances across 80 scenes, 16 edit types in four semantic categories, and captions at three verbosity levels, with an average of only 8% pixel change between base and foil images. Evaluation of 22 dual‑encoder VLMs shows that all models perform at or below chance on CompoSET, contrasting with their strong performance on prior benchmarks that conflate global image differences with compositional reasoning.

By Laure Delisle, Pietro Perona