arXiv Computer Vision
MLLM-Assisted Audio VOS: A 3rd Place Report for the MeViS-Audio Track, 8th LSVOS Challenge
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Aug 27
ARGenSeg: Image Segmentation with Autoregressive Image Generation Model
ARGenSeg introduces an autoregressive generation-based approach for image segmentation that integrates seamlessly with multimodal large language models (MLLMs). Unlike prior methods that use boundary points or dedicated segmentation heads, ARGenSeg generates dense masks directly through visual token output and detokenization via a universal VQ‑VAE, enabling fine‑grained pixel‑level perception. The framework employs a next‑scale‑prediction strategy to parallelize token generation, resulting in faster inference while outperforming state‑of‑the‑art segmentation models on multiple datasets.
arXiv AI
Jun 3
AVTrack: Audio-Visual Tracking in Human-centric Complex Scenes
arXiv:2606. 02724v1 Announce Type: cross Abstract: Audio-visual speaker tracking aims to localize and track active speakers by leveraging auditory and visual cues, enabling fine-grained, human-centric scene understanding.
arXiv Computer Vision
Aug 25
Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videos
arXiv:2501.04001v4 Announce Type: replace Abstract: This work presents Sa2VA, the first comprehensive, unified model for dense grounded understanding of both images and videos. Unlike existing multi-...
arXiv Computer Vision
Aug 25
Motion-Aware Reasoning from Speech to Mask Tracks: Runner-up Solution for the MeViS-Audio Track of the 8th LSVOS Challenge 2026
arXiv:2608.22337v1 Announce Type: cross Abstract: Speech-guided referring video object segmentation aims to recover the mask tracks of objects specified by a spoken motion description. Here, speech c...
arXiv Computer Vision
Sep 3
Video Object Segmentation-Aware Audio Generation
The paper introduces a new task called video object segmentation‑aware audio generation, which conditions sound synthesis on object‑level segmentation maps. It presents SAGANet, a multimodal generative model that uses visual segmentation masks, video, and textual cues to produce controllable audio for musical instruments, offering fine‑grained, visually localized control. The authors also release the Segmented Music Solos dataset of instrument performance videos with segmentation information to support this task and demonstrate that SAGANet outperforms current state‑of‑the‑art methods in controllable, high‑fidelity Foley synthesis.
Hugging Face Trending Papers
Sep 1
From Visual Cues to Spoken Narration: Rethinking Audio Description
Audio Description (AD) provides spoken narration of visual events during dialogue gaps, making movies accessible to visually impaired audiences. The problem requires determining both what (which visua...