arXiv Computer Vision
Adapting Personalized Speech Enhancement for Low-Latency Audio-Visual Target-Speaker Extraction
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Aug 11
DAVE: A Decoupled Audio-Visual Enhancement Framework for Real-World Speech Separation
arXiv:2608. 09288v1 Announce Type: cross Abstract: Audio-visual speech enhancement under real-world conditions remains challenging due to unreliable visual inputs and the lack of large-scale training data with realistic acoustic conditions.
arXiv Computation and Language
Sep 23
Challenges of Multi-Speaker Extraction for Real Conversational Speech Enhancement
The paper addresses challenges in extracting target and multiple speakers from real conversational speech, noting that real conversations contain more silence and enrolment samples that differ from the target speech. It introduces a new loss function that reduces the impact of excess silence during training, yielding improvements in STOI (from 0.55 to 0.60) and frequency‑weighted segmental SNR (from 4.35 to 5.12). The study also investigates how mismatches between enrolment and target speech affect performance.
arXiv Machine Learning
Sep 22
AVTR-1: Open Stack for Real-Time Interactive Avatars
arXiv:2609.22913v1 Announce Type: cross Abstract: Talking-head and dyadic models now achieve real-time inference, yet fast motion generation alone does not produce an interactive conversation. A live...
Hugging Face Trending Papers
Sep 8
ConversationalVoice: Full-Duplex Speech Data from Real Conversations through Source-Faithful Reconstruction and Conversation-Grounded Expansion
Full-duplex speech models require training data that preserves turn-taking, overlap, interruption, and backchannel behavior, yet these signals are entangled across speakers in noisy real-world recordi...
arXiv Computer Vision
Sep 10
AVSRBench: A Multi-Condition AVSR Benchmark
arXiv:2609.10366v1 Announce Type: cross Abstract: While AVSR has achieved sub-1% word error rates on the standard LRS3 benchmark, its reliance on broadcast speech obscures whether this reflects true...
arXiv Machine Learning
Jun 16
SSNAPS: Audio-Visual Separation of Speech and Background Noise with Diffusion Inverse Sampling
arXiv:2602. 01394v2 Announce Type: replace-cross Abstract: This paper addresses the challenge of audio-visual single-microphone speech separation and enhancement in the presence of real-world environmental noise.