arXiv Computation and Language

When Forgetting Looks Like Improvement: Metric Masking in Streaming Diarizer Adaptation and the Price of Rehearsal

arXiv AI
Sep 24

Forget who you Forgot: Speaker Unlearning to Prevent Re-Identification in Zero-Shot Text-to-Speech

The paper introduces GUARD, a lightweight speaker identity unlearning framework designed to prevent re-identification in zero-shot text-to-speech systems. GUARD employs a learned speaker gate and speaker-agnostic activation steering on a frozen TTS backbone, optimizing steering vectors through group-relative reward optimization to reduce similarity to forgotten speakers while maintaining intelligibility and naturalness. Experiments on CosyVoice2 show that GUARD significantly lowers forget-speaker similarity and re-identification accuracy while preserving the ability to reproduce retained speakers.

By Hyoeun Kim, Yujun Lee, Kyuhong Shim
arXiv Computation and Language
Sep 25

Interactive In-Meeting Speaker Correction with Human Feedback

The paper introduces an LLM-assisted system for correcting speaker attribution errors during meetings. It combines streaming ASR, diarization, and concise LLM-generated summaries to guide users in providing brief corrective feedback, which updates the transcript and adds online speaker enrollments. The approach includes mechanisms to accurately interpret user corrections and a simulation for large-scale evaluation, achieving significant reductions in DER and speaker substitution error on the AMI headset test set, with a pilot usability study highlighting further improvements.

By Xinlu He, Yiwen Guan, Badrivishal Paurana, Pitipat Kongsomjit, Zilin Dai, Jacob Whitehill