arXiv Computer Vision
Binaural Audio-Visual Instance Segmentation
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Jun 3
AVTrack: Audio-Visual Tracking in Human-centric Complex Scenes
arXiv:2606. 02724v1 Announce Type: cross Abstract: Audio-visual speaker tracking aims to localize and track active speakers by leveraging auditory and visual cues, enabling fine-grained, human-centric scene understanding.
arXiv AI
Aug 18
Audio-Visual Segmentation via Depth-Guided Collaborative Modeling
arXiv:2608. 16285v1 Announce Type: cross Abstract: Audio-Visual Segmentation (AVS) is a fundamental task in multimodal perception that performs pixel-level segmentation of sounding objects in videos by leveraging both visual and audio cues.
arXiv AI
Jul 17
SceneBind: Binding What and Where Across Vision, Audio and Language
arXiv:2607. 15265v1 Announce Type: cross Abstract: We present SceneBind, an omni-modal representation of realistic scenes with joint semantic and 3D spatial understanding across vision, audio and language.
arXiv Computer Vision
6d ago
Semantic-Critical Regions for Video-to-Audio Generation
The paper introduces SCR-V2A, a video-to-audio generation framework that focuses on Semantic-Critical Regions (SCRs)—localized visual cues directly related to sound events. SCR-V2A first identifies these regions via sound-related visual localization, then encodes them with an adapter-based dual visual encoder to produce a control representation that captures both global context and fine-grained sound evidence. This representation conditions an audio generation model, and experiments on in-domain and out-of-domain benchmarks show improved distribution matching and audio quality with few trainable parameters.
arXiv AI
Jul 20
Acoustic Imaging for UAV Detection: Dense Beamformed Energy Maps and U-Net SELD
arXiv:2508. 00307v4 Announce Type: replace-cross Abstract: We introduce a U-net model for 360{\deg} acoustic source localization formulated as a spherical semantic segmentation task.
arXiv Computer Vision
4d ago
Frequency-Domain Decomposition and Recomposition for Robust Audio-Visual Segmentation
The paper introduces Frequency-Aware Audio-Visual Segmentation (FAVS), a framework that treats audio-visual segmentation as a frequency-domain decomposition and recomposition task. It comprises a Frequency-Domain Enhanced Decomposer (FDED) that iteratively separates modality-specific semantics and structural features, and a Synergistic Cross-Modal Consistency (SCMC) module that uses a mixture-of-experts architecture to maintain semantic consistency and preserve modality-specific features. Experiments on three benchmark datasets show that FAVS achieves state-of-the-art performance, with qualitative visualizations supporting the effectiveness of both modules.