DRAgent introduces a discriminative reasoning framework for Referring Expression Segmentation that replaces direct coordinate prediction with a two‑stage selection process. First, a detector generates candidate boxes; then a multimodal large language model (MLLM) discriminates the correct target through high‑recall screening and instance‑wise verification. The chosen box serves as a spatial prompt for a foundation segmentation model, and a self‑consistency‑filtered data pipeline is used for LoRA‑based fine‑tuning, yielding competitive results on RefCOCO, RefCOCO+, and RefCOCOg.
Referring Expression Segmentation (RES) aims to generate a pixel-level mask for the object specified by a language expression. Recent methods based on multimodal large language models (MLLMs) often re...
The paper introduces Bidirectional Reciprocal Learning (BRL), a parameter‑efficient fine‑tuning framework for referring image segmentation that operates on frozen vision foundation models. BRL employs two lightweight adapters—Reciprocal Attention Adapter (RAA) for token‑level cross‑modal attention and Reciprocal Gate Adapter (RGA) for channel‑level gating—to enable hierarchical, bidirectional information flow between vision and language. Experiments on RefCOCO, RefCOCO+, and RefCOCOg show that BRL outperforms existing methods while updating fewer than 0.5% of backbone parameters.
arXiv:2609.24510v2 Announce Type: replace Abstract: Recent advances in vision foundation models (VFMs) have shown remarkable capabilities across diverse unimodal visual tasks. However, adapting VFMs...
arXiv:2610.20352v1 Announce Type: new Abstract: Multimodal Large Language Models have made significant progress in referring expression comprehension (REC), yet most REC benchmarks still focus on fin...
arXiv:2512. 06276v3 Announce Type: replace-cross Abstract: Referring Expression Comprehension (REC) is a vision-language task that localizes a specific image region based on a textual description.
PSGAgent is an agentic framework that improves Multimodal Large Language Models’ performance on Panoptic Scene Graph generation by enabling dynamic visual perception. It introduces a think‑act‑observe loop with a Region Zoom‑In Tool for detailed local exploration and a Visual Highlight Tool for focused segmentation, reducing noise from irrelevant context. A two‑stage training pipeline—first building a high‑quality, error‑aware Multimodal Chain‑of‑Thought dataset, then applying a hierarchical reward mechanism with Group Sequence Policy Optimization—optimizes tool usage and achieves state‑of‑the‑art results on the PSG benchmark.
SetCon reformulates open-ended referring segmentation as set-level concept prediction, using LVLM-generated natural-language concepts instead of segmentation-specific tokens. It employs a hierarchical semantic decomposition that first predicts a shared set-level concept and then refines it into fine-grained concept groups aligned with target subsets. The method, supported by a two-stage annotation pipeline adding hierarchical semantic supervision, achieves state-of-the-art results on multiple image and video benchmarks, with performance gains increasing as the number of referred targets grows.
The paper introduces FindIt, the first comprehensive benchmark for evaluating the promptable localization abilities of generalist multimodal large language models (MLLMs). It covers four core task categories—object detection, referring expression detection, instance-level detection, and video-based detection—and provides a unified framework that standardizes inputs, enforces parsable bounding box outputs, and defines transparent evaluation protocols. Using this benchmark, the authors assess a range of open-source and proprietary MLLMs, revealing both their strengths and limitations, particularly their sensitivity to formatting constraints and difficulty generalizing to minor variations.
The paper introduces MK‑FSS, a few‑shot segmentation framework that leverages Multimodal Large Language Models (MLLMs) to extract spatial and semantic target knowledge from query images. Spatial knowledge is encoded into a memory representation and fused with support‑guided memory via a dual‑memory debate‑fusion module, while semantic knowledge is turned into a textual feature and combined with multi‑scale query features through a progressive cross‑modal prompt generator. Together, these components produce a robust target representation that improves segmentation performance over existing methods.
arXiv:2606. 00987v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have shown strong visual understanding and language-guided grounding abilities, yet their capacity for multi-temporal visual reasoning remains underexplored.
arXiv:2501.04001v4 Announce Type: replace Abstract: This work presents Sa2VA, the first comprehensive, unified model for dense grounded understanding of both images and videos. Unlike existing multi-...
arXiv:2606. 17539v1 Announce Type: cross Abstract: Spatial VLMs have made substantial progress in geometric perception, yet complex spatial reasoning requiring multi-step inference over depth, distance, and scene relations remains challenging.