VM-ARRAYDPS introduces virtual microphones with higher signal‑to‑noise ratios to augment a limited physical microphone array, providing additional multi‑channel consistency constraints for blind speech separation. By leveraging a pretrained speech diffusion model and posterior sampling, the method improves upon the existing ArrayDPS approach. Experiments on 2‑speaker and 3‑speaker datasets show significant performance gains, and ablation studies reveal the impact of virtual microphone count and consistency weighting.
By Jingqi Sun, Haozhan Tang, Shulin He, Zhong-Qiu Wang
arXiv:2602. 01394v2 Announce Type: replace-cross Abstract: This paper addresses the challenge of audio-visual single-microphone speech separation and enhancement in the presence of real-world environmental noise.
By Yochai Yemini, Yoav Ellinson, Rami Ben-Ari, Sharon Gannot, Ethan Fetaya
arXiv:2609.07226v1 Announce Type: cross
Abstract: This paper proposes a general framework for stable and effective iterative audio separation with mixture consistency by extending source separation m...
By Yukara Ikemiya, WeiHsiang Liao, Yuki Mitsufuji
The paper introduces CF-AVSS, a two‑stage audio‑visual speech separation framework that first generates coarse estimates using a synchronized visual‑stream SPAV‑TFGridNet, then refines these estimates with a TFGridNet‑based target‑speaker refinement network. The final output is a fusion of coarse and refined estimates calibrated on a development set. In the ISCSLP 2026 Real‑World AVSE Challenge, the system tied for first place on Track 1 and ranked second on Track 2, achieving the lowest character error rates of 12.31 % and 14.79 % respectively.
By Longjie Luo, Zhongshu Hou, Yuepeng Li, Gang Yang, Shidong Shang, Fan Zhou
arXiv:2609.30631v1 Announce Type: cross
Abstract: Online audio-visual target-speaker extraction aims to remove competing voices while preserving speech quality and bounding lookahead. Existing extrac...
By Rayhan Rashed, Senja Filipi, Ross Cutler
arXiv:2606. 23712v1 Announce Type: cross Abstract: Audio-visual speech enhancement (AVSE) exploits visual cues such as lip movements to recover speech in noisy environments.
By Colombe Mboungou (MULTISPEECH), Mostafa Sadeghi (MULTISPEECH), Jean-Eudes Ayilo (MULTISPEECH), Romain Serizel (MULTISPEECH)
arXiv:2608. 09288v1 Announce Type: cross Abstract: Audio-visual speech enhancement under real-world conditions remains challenging due to unreliable visual inputs and the lack of large-scale training data with realistic acoustic conditions.
By Wei Zhou, Wanyi Ning, Yinshang Guo, Qianxiao Fang, Haitao Qian, Yingpeng Li
Audio-visual speech enhancement (AVSE) leverages visual cues to recover target speech from mixtures corrupted by reverberation, noise, and overlapping speakers. We propose a coarse-to-fine two-stage A...
arXiv:2508. 14623v2 Announce Type: replace-cross Abstract: This paper examines the implications of using the Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) as both evaluation and training objective in supervised speech separation, when the training references contain noise, as is the case with the de facto benchmark WSJ0-2Mix.
By Simon Dahl Jepsen, Mads Gr{\ae}sb{\o}ll Christensen, Jesper Rindom Jensen
arXiv:2607. 23395v1 Announce Type: cross Abstract: Music Source Separation (MSS), the task of recovering individual sound components (stems) from a polyphonic mixture, is central to applications ranging from karaoke and remixing to audio restoration and content production.
By Roman Solovyev, Ilya Kiselev, Alexander Stempkovskiy, Tatiana Gabruseva
arXiv:2606. 09677v1 Announce Type: cross Abstract: While discriminative models for multi-channel speech separation excel in reference-based metrics, they often exhibit suboptimal human listening quality.
By Dohwan Kim, Jung-Woo Choi
arXiv:2604. 24199v4 Announce Type: replace-cross Abstract: We propose Speech Enhancement based on Drifting Models (DriftSE), a novel generative framework that formulates denoising as an equilibrium problem.
By Liang Xu, Diego Caviedes-Nozal, W. Bastiaan Kleijn, Longfei Felix Yan, Rasmus Kongsgaard Olsson