Hugging Face Trending Papers
PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Computation and Language
Sep 17
PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection
PANORAMA introduces a new panoptic grounded captioning framework that jointly generates detailed image captions and associates each phrase with precise pixel-level masks. The authors create PanoCaps, a human‑annotated benchmark with dense captions and near‑complete pixel coverage, and propose a phrase‑mask matching protocol with a generalized Panoptic Quality metric. PANORAMA conditions a pretrained segmenter on contextualized phrase representations, learns to select appropriate masks, and achieves state‑of‑the‑art grounding performance on PanoCaps and other pixel‑level tasks.
arXiv AI
Aug 11
Vision-Language Grounding as Bidirectional Concept Correspondence
arXiv:2608. 07886v1 Announce Type: cross Abstract: Vision-language grounding connects language to visual content, yet most existing formulations reduce grounding to a unidirectional localization problem: given a prespecified text phrase or category name, identify the corresponding image region.
arXiv Computer Vision
Oct 26
SetCon: Towards Open-Ended Referring Segmentation via Set-Level Concept Prediction
SetCon reformulates open-ended referring segmentation as set-level concept prediction, using LVLM-generated natural-language concepts instead of segmentation-specific tokens. It employs a hierarchical semantic decomposition that first predicts a shared set-level concept and then refines it into fine-grained concept groups aligned with target subsets. The method, supported by a two-stage annotation pipeline adding hierarchical semantic supervision, achieves state-of-the-art results on multiple image and video benchmarks, with performance gains increasing as the number of referred targets grows.
arXiv Computer Vision
Aug 25
Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videos
arXiv:2501.04001v4 Announce Type: replace Abstract: This work presents Sa2VA, the first comprehensive, unified model for dense grounded understanding of both images and videos. Unlike existing multi-...
arXiv AI
Aug 13
Learning from Multimodal Pseudo-Labels for Robust Open-Vocabulary Instance and Panoptic Segmentation
arXiv:2608. 11681v1 Announce Type: cross Abstract: This work addresses the challenge of open-vocabulary instance segmentation (OVIS) and open-set panoptic segmentation (OSPS), which aim to recognize both predefined and unseen object categories without exhaustive human annotations.
arXiv Computer Vision
Oct 9
PixVL: Self-Supervised Training of Pixel-Level MLLMs via a Unified Mask--Text Consistency Cycle
PixVL introduces a self‑supervised post‑training framework for pixel‑level multimodal large language models, addressing the scarcity of mask‑text pairs and optimization interference between region segmentation and region understanding. It employs a unified Mask‑Text Consistency Cycle that generates and self‑verifies regional descriptions using confuser‑aware semantic verification and cross‑view verification, while a quality‑coupled bidirectional learning strategy turns segmentation and understanding into mutual generators and verifiers. Experiments show that PixVL improves performance on both region understanding and segmentation tasks.