DRAgent introduces a discriminative reasoning framework for Referring Expression Segmentation that replaces direct coordinate prediction with a two‑stage selection process. First, a detector generates candidate boxes; then a multimodal large language model (MLLM) discriminates the correct target through high‑recall screening and instance‑wise verification. The chosen box serves as a spatial prompt for a foundation segmentation model, and a self‑consistency‑filtered data pipeline is used for LoRA‑based fine‑tuning, yielding competitive results on RefCOCO, RefCOCO+, and RefCOCOg.
By Yujie Qi, Luyan Zhang
arXiv:2512. 06276v3 Announce Type: replace-cross Abstract: Referring Expression Comprehension (REC) is a vision-language task that localizes a specific image region based on a textual description.
By Tianyi Gao, Hao Li, Han Fang, Xin Wei, Xiaodong Dong, Hongbo Sun, Ye Yuan, Zhongjiang He, Jinglin Xu, Jingmin Xin, Hao Sun
The paper introduces Bidirectional Reciprocal Learning (BRL), a parameter‑efficient fine‑tuning framework for referring image segmentation that operates on frozen vision foundation models. BRL employs two lightweight adapters—Reciprocal Attention Adapter (RAA) for token‑level cross‑modal attention and Reciprocal Gate Adapter (RGA) for channel‑level gating—to enable hierarchical, bidirectional information flow between vision and language. Experiments on RefCOCO, RefCOCO+, and RefCOCOg show that BRL outperforms existing methods while updating fewer than 0.5% of backbone parameters.
By Xiaoqiang Lu, Licheng Jiao, Lingling Li, Yuting Yang, Long Sun, Wenping Ma, Xu Liu, Fang Liu
arXiv:2609.24510v2 Announce Type: replace
Abstract: Recent advances in vision foundation models (VFMs) have shown remarkable capabilities across diverse unimodal visual tasks. However, adapting VFMs...
By Xiaoqiang Lu, Licheng Jiao, Lingling Li, Yuting Yang, Long Sun, Wenping Ma, Xu Liu, Fang Liu
arXiv:2610.20352v1 Announce Type: new
Abstract: Multimodal Large Language Models have made significant progress in referring expression comprehension (REC), yet most REC benchmarks still focus on fin...
By Tianyi Gao, Han Fang, Xin Wei, Yaoyao Zhong, Zhaoyan Pan, Weijie Zhu, Hongbo Sun, Zhiyu Lin, Kaicheng Yang, Jinglin Xu, Ye Yuan, Hao Li, Hao Sun, Jingmin Xin
SetCon reformulates open-ended referring segmentation as set-level concept prediction, using LVLM-generated natural-language concepts instead of segmentation-specific tokens. It employs a hierarchical semantic decomposition that first predicts a shared set-level concept and then refines it into fine-grained concept groups aligned with target subsets. The method, supported by a two-stage annotation pipeline adding hierarchical semantic supervision, achieves state-of-the-art results on multiple image and video benchmarks, with performance gains increasing as the number of referred targets grows.
By Zhixiong Zhang, Yizhuo Li, Shuangrui Ding, Yuhang Zang, Shengyuan Ding, Long Xing, Yibin Wang, Qiaosheng Zhang, Jiaqi Wang
arXiv:2606. 00987v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have shown strong visual understanding and language-guided grounding abilities, yet their capacity for multi-temporal visual reasoning remains underexplored.
By Bingyu Li, Da Zhang, Tao Huo, Zhiyuan Zhao, Junyu Gao, Xuelong Li
PSGAgent is an agentic framework that improves Multimodal Large Language Models’ performance on Panoptic Scene Graph generation by enabling dynamic visual perception. It introduces a think‑act‑observe loop with a Region Zoom‑In Tool for detailed local exploration and a Visual Highlight Tool for focused segmentation, reducing noise from irrelevant context. A two‑stage training pipeline—first building a high‑quality, error‑aware Multimodal Chain‑of‑Thought dataset, then applying a hierarchical reward mechanism with Group Sequence Policy Optimization—optimizes tool usage and achieves state‑of‑the‑art results on the PSG benchmark.
By Qingyun Liu, Yuankun Liu, Jiaye Feng, Jie Huang, Zonghong Dai, Haochen Li, Jingkun Wang, Weiping Li, Tong Mo
MV-STRIDE is a Multi‑View hierarchical Spatial Reasoning dataset that models dependencies among perception, scene understanding, and contextual reasoning to support 3D spatial cognition. It introduces a QA generation pipeline that enforces cross‑view constraints, producing multi‑level reasoning tasks with chain‑of‑thought supervision. Experiments show that training on MV‑STRIDE yields state‑of‑the‑art performance on multi‑view spatial benchmarks, enabling MLLMs to reason robustly across diverse viewpoints.
By Jin Xu, Xiaojian Huang, Zhuodong Luo, Zhihong Zhang, Xin Liu, Jiansheng Wei, Xinzhi Wang, Jie Zhao, Xuejin Chen
The paper introduces MK‑FSS, a few‑shot segmentation framework that leverages Multimodal Large Language Models (MLLMs) to extract spatial and semantic target knowledge from query images. Spatial knowledge is encoded into a memory representation and fused with support‑guided memory via a dual‑memory debate‑fusion module, while semantic knowledge is turned into a textual feature and combined with multi‑scale query features through a progressive cross‑modal prompt generator. Together, these components produce a robust target representation that improves segmentation performance over existing methods.
By Yijun Hu, Heng Fan, Libo Zhang
The paper introduces DRMOT, a new dataset and framework for Referring Multi-Object Tracking (RMOT) that incorporates RGB-D data. DRSet contains 2,076 referring expressions across 186 RGB-D sequences, pairing synchronized RGB images, depth maps, and language descriptions—including depth-dependent cues—to evaluate spatial‑semantic detection and tracking. The proposed DRTrack framework uses a Geometric‑Aware GRPO fine‑tuned Multimodal Large Language Model to perform precise detection and robust trajectory association from joint language, RGB, and depth inputs, with experiments demonstrating its effectiveness.
By Sijia Chen, Shaofeng Liang, Lijuan Ma, Yanqiu Yu, Nan Li, Wenshuo Chen, Runwei Guan, Dongming Wu, En Yu, Yutao Yue
arXiv:2501.04001v4 Announce Type: replace
Abstract: This work presents Sa2VA, the first comprehensive, unified model for dense grounded understanding of both images and videos. Unlike existing multi-...
By Haobo Yuan, Xiangtai Li, Tao Zhang, Yueyi Sun, Zilong Huang, Shilin Xu, Shunping Ji, Yunhai Tong, Lu Qi, Jiashi Feng, Ming-Hsuan Yang