← Back to all news
arXiv Computer Vision By Kai Peng, Yunzhe Shen, Miao Zhang, Jia Wang, Leiye Liu, Wei Ji, Jingjing Li, Yongri Piao, Huchuan Lu

Frequency-Domain Decomposition and Recomposition for Robust Audio-Visual Segmentation

Read the original on arXiv Computer Vision →

The paper introduces Frequency-Aware Audio-Visual Segmentation (FAVS), a framework that treats audio-visual segmentation as a frequency-domain decomposition and recomposition task. It comprises a Frequency-Domain Enhanced Decomposer (FDED) that iteratively separates modality-specific semantics and structural features, and a Synergistic Cross-Modal Consistency (SCMC) module that uses a mixture-of-experts architecture to maintain semantic consistency and preserve modality-specific features. Experiments on three benchmark datasets show that FAVS achieves state-of-the-art performance, with qualitative visualizations supporting the effectiveness of both modules.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Machine Learning
Jul 14

Empowering Long-form Omni-modal Understanding with Robust Audio Perception

arXiv:2607. 10299v1 Announce Type: new Abstract: Recent advances in large-scale multimodal models have drivenremarkable progress in vision-language tasks; however, comprehensiveomni-modal understanding remains under-explored, largely due to thescarcity of datasets with rich, explicitly aligned auditory cues.

By Kaiying Yan, Luoyi Sun, Xiao Zhou, Weidi Xie
fine-tuningmultimodalbenchmarks
More like this →
arXiv Computer Vision
Oct 12

Shift-Aware Identity-Guided Latent Refinement for Referring Audio--Visual Segmentation

The paper introduces Shift-Aware Identity-guided Latent (SAIL) refinement for Referring Audio‑Visual Segmentation (Ref‑AVS), a task that requires precise segmentation of a specific object using both visual and auditory cues grounded in natural language. SAIL employs a dual‑token strategy to separate frame‑level dynamics from a global semantic anchor, a shift‑aware refinement module to correct target‑shift drift, and a decoder with continuous memory updates to preserve spatio‑temporal coherence and identity consistency. Experiments on two Ref‑AVS benchmarks show that SAIL outperforms existing AVS, Ref‑VOS, and Ref‑AVS methods.

By Kun Li, Sami Sebastian Brandt, Michael Ying Yang
ragcomputer-visionmultimodalbenchmarks
More like this →