arXiv AI
Speech Generation Speaker Poisoning: Capability Erasure in Zero-Shot Text-to-Speech
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 19
ZeSTA: Zero-Shot TTS Augmentation with Domain-Conditioned Training for Data-Efficient Personalized Speech Synthesis
arXiv:2603. 04219v2 Announce Type: replace-cross Abstract: We investigate the use of zero-shot text-to-speech (ZS-TTS) as a data augmentation source for low-resource personalized speech synthesis.
arXiv AI
Aug 18
Adding Voice Cloning to Text-to-Audio-Video Models with a Single Zero-Initialised Layer
arXiv:2608. 15690v1 Announce Type: cross Abstract: Text-to-audio-video (T2AV) generation models produce a video and its soundtrack from a textual description, but offer no control over whose voice speaks in the output.
arXiv AI
Jul 7
NouveauVoice: Generating Novel Pseudo Speakers for Voice Anonymization
arXiv:2607. 03985v1 Announce Type: cross Abstract: Advanced neural technologies in speech synthesis and voice conversion (VC) have introduced severe risks to personal privacy, necessitating robust Speaker Anonymization Systems (SAS).
arXiv AI
Jun 26
VoiceTTA: Enhancing Zero-Shot Text-to-Speech via Reinforcement Learning-Based Test-Time Adaptation
arXiv:2606. 26534v1 Announce Type: cross Abstract: Recently, zero-shot text-to-speech (TTS) has enabled high-fidelity and expressive speech synthesis, but it often fails to imitate unseen speaking styles from uncommon scenarios (e.
arXiv AI
Sep 7
X-VC: Zero-shot Streaming Voice Conversion in Codec Space
X-VC is a zero‑shot streaming voice conversion system that performs one‑step conversion directly in the latent space of a pretrained neural codec. It employs a dual‑conditioning acoustic converter that jointly models source codec latents and target acoustic conditions, while using adaptive normalization to inject utterance‑level speaker information. The model is trained with generated paired data and a role‑assignment strategy, and uses a chunkwise inference scheme with overlap smoothing to achieve low‑latency streaming inference, achieving superior WER, speaker similarity, and real‑time factor on the Seed‑TTS‑Eval benchmark.
arXiv Machine Learning
Sep 3
Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Models
The paper introduces a black-box membership inference attack framework tailored for fine-tuned text-to-speech models, addressing challenges in query generation and representation engineering. It evaluates five query types, finding recitation queries most effective, and uses multi-level speech embeddings with temporal alignment for fine-grained comparison. Experiments on CosyVoice2, F5-TTS, and XTTS-v2 trained on VCTK and British Dialect datasets show high privacy leakage, with speaker-level AUC above 0.80 and record-level AUC between 0.80 and 0.90.