The paper introduces DCR‑SAM, a framework that adapts the Segment Anything Model 3 (SAM3) for multi‑class semantic segmentation. By decoupling cross‑modal interactions, aligning textual concepts with visual representations, and adding a dense competition head and learnable background token, DCR‑SAM reduces inference latency, mitigates semantic misclassifications, and suppresses overlapping predictions. Experiments show 2.3%–10.7% performance gains, a 33× speedup over native SAM3, and strong robustness across benchmarks.
The Segment Anything Model 3 (SAM3) has achieved significant progress in Promptable Concept Segmentation (PCS) by processing short noun phrases to generate segmentation masks with unique instance iden...
UnSAMv2 is a self‑supervised extension of the Segment Anything Model that can segment objects at any granularity without human annotations. It discovers mask‑granularity pairs and introduces a granularity control embedding for continuous scale adjustment, requiring only 6K unlabeled images and 0.02% extra parameters. The model outperforms SAM/SAM‑2 across 11+ benchmarks, improving metrics such as 1‑IoU from 58.0 to 73.1 and achieving state‑of‑the‑art performance on language‑guided segmentation.
arXiv:2606.16996v2 Announce Type: replace-cross Abstract: Segment Anything Model 3 (SAM 3) provides a strong frozen backbone for concept-prompted segmentation, but applying it directly to open-vocabu...
ProCAP introduces a probabilistic cross-attentive prompt learning framework for vision-language models like CLIP, enabling improved cross-modal interaction without updating the backbone. It jointly learns visual and textual prompt tokens, linking them via stacked bidirectional multi-head cross-attention to refine each branch across prompt depth. The method incorporates Gaussian parameterization of prompt tokens, lightweight KL and L2 regularization, and a compact symmetric InfoNCE head to align image features with class-level text representations, achieving strong few-shot base-to-novel performance and competitive transfer results across multiple datasets and benchmarks.
arXiv:2501.04001v4 Announce Type: replace Abstract: This work presents Sa2VA, the first comprehensive, unified model for dense grounded understanding of both images and videos. Unlike existing multi-...
arXiv:2609.36680v1 Announce Type: new Abstract: Visual reprogramming adapts pretrained models to downstream tasks by modifying their input and output interfaces while keeping the backbone fixed. In v...
The paper introduces Language-driven Dense Semantic Adaptor (LDSA) for multi-label image classification with incomplete annotations. LDSA leverages multimodal pretrained CLIP models to extract prior-adaptive relationships, employing a densely contrastive adaptor for visual contrastive constraints and a language-driven interactive decoder with class-specific prompt tuning. Experiments show LDSA achieves state‑of‑the‑art performance on public benchmarks and reveals implicit semantic relationships through its learning scheme.
arXiv:2607. 00684v1 Announce Type: new Abstract: The classification accuracy of pretrained Vision-Language Models (VLMs) relies on the quality of the text prompts.
arXiv:2602. 21397v2 Announce Type: replace-cross Abstract: Prompt learning has become a dominant paradigm for adapting vision-language models (VLMs) such as CLIP to downstream tasks without modifying pretrained weights.
arXiv:2609.16722v1 Announce Type: new Abstract: Scaling Multimodal Large Language Models (MLLMs) to long-form video understanding is bottlenecked by the explosion of visual tokens, which saturates co...
TokenSwap introduces a method to benchmark and reduce the modality gap in multimodal large language models (MLLMs) by interleaving semantically aligned images with text tokens. By transforming text-only benchmarks like MMLU into image‑interleaved versions (TokenSwap‑Bench), the authors reveal a pervasive performance drop—averaging 19.6%—across 42 MLLMs, with reasoning models showing smaller gaps. Training with TokenSwap mitigates this gap while maintaining strong text‑only and vision‑language performance.
Prompt-driven vision-language models (VLMs) hold immense promise for accelerating dense remote sensing (RS) annotation, but static models suffer from severe performance degradation when deployed on novel scenes, unseen categories, or visually confusing backgrounds. Moreover, existing unified paradigms primarily rely on intra-image specific prompts, lacking flexible task routing to adapt to multi-intent operational workflows.