The paper introduces a two‑stage speech anonymization framework that preserves both linguistic content and acoustic identity. It replaces personally identifiable information using a generative editing model and applies a flow‑matching anonymization technique (F3‑VA) to create diverse, distinct anonymized speakers. The authors evaluate privacy with speaker verification metrics and utility by training ASR, TTS, and SER models from scratch, showing stronger privacy protection with minimal utility loss compared to existing baselines.
arXiv:2607. 03985v1 Announce Type: cross Abstract: Advanced neural technologies in speech synthesis and voice conversion (VC) have introduced severe risks to personal privacy, necessitating robust Speaker Anonymization Systems (SAS).
The paper introduces VoxPrivacy, a benchmark for assessing interactional privacy in Speech Language Models (SLMs). It evaluates models on a 32‑hour bilingual dataset across three difficulty tiers, revealing that most open‑source SLMs perform near random on conditional privacy decisions and even strong closed‑source systems struggle with proactive privacy inference. The authors also validate these findings on a real‑speech subset and show that fine‑tuning on a 4,000‑hour training set can improve privacy‑preserving capabilities while maintaining robustness.
DiffAnon is a diffusion‑based voice anonymization method that uses classifier‑free guidance to give users continuous, inference‑time control over how much prosody is preserved. By refining acoustic detail over semantic embeddings from an RVQ codec, the model allows smooth interpolation between strong anonymization and high prosodic fidelity within a single architecture. This is the first framework to provide structured, interpolatable prosody control while maintaining competitive privacy and utility across different operating points.
arXiv:2607. 09767v1 Announce Type: cross Abstract: The paper presents a voice anonymization model focusing on preserving content rather than producing realistic speech.
The paper demonstrates that the multilingual voice cloning model XTTSv2 can be repurposed for speaker anonymization without retraining. By conditioning on a pseudo-speaker and using an iterative refinement strategy, the authors balance privacy and intelligibility, achieving near‑optimal privacy (EER ≈ 0.49) and competitive speech quality across seven European languages. The method outperforms dedicated anonymization baselines and requires no language‑specific training.
arXiv:2606. 29897v1 Announce Type: cross Abstract: Voice anonymization aims to protect speaker identity while preserving linguistic content and speech usability.
The paper investigates how acoustic and content-oriented speaker verification attacks perform against multilingual voice anonymization. It evaluates both attacker types on anonymized speech, builds a multilingual voice-converted dataset to enhance cross‑lingual generalization, and finds that attack success depends on how much linguistic utility is preserved. Acoustic‑oriented attacks generally outperform content‑oriented ones, but the performance gap narrows when linguistic information is well maintained, and the new dataset improves results while reducing cross‑lingual disparities.
The paper examines whether word error rate adequately measures privacy in speech by applying entity‑aware metrics from NLP to evaluate two obfuscation techniques. It focuses on protecting named entities and explores how fine‑tuning on entity‑rich data affects attack success across different entity categories. The authors also offer guidance on choosing metrics depending on whether the obfuscation preserves temporal alignment.
The paper introduces a black-box membership inference attack framework tailored for fine-tuned text-to-speech models, addressing challenges in query generation and representation engineering. It evaluates five query types, finding recitation queries most effective, and uses multi-level speech embeddings with temporal alignment for fine-grained comparison. Experiments on CosyVoice2, F5-TTS, and XTTS-v2 trained on VCTK and British Dialect datasets show high privacy leakage, with speaker-level AUC above 0.80 and record-level AUC between 0.80 and 0.90.
The paper systematically studies how anonymizing input data affects large language models (LLMs). Five prominent LLMs were evaluated on eleven benchmarks, comparing performance on original versus pseudonymized inputs. Results show that anonymization generally degrades performance, with larger drops for more capable models and task-dependent effects; reversible anonymization preserves entity uniqueness better than irreversible redaction, and prompting about anonymization offers no benefit.
arXiv:2606. 24941v2 Announce Type: replace-cross Abstract: Reviewing recorded interviews for affective cues such as composure and agitation is slow and subjective, and cloud services that could automate the task require sensitive audio to leave the device.
PHONOS is a real‑time streaming module for speaker anonymization that neutralizes accent cues by converting non‑native segmental realizations toward a target accent domain. It uses pre‑generated golden utterances that preserve timbre and rhythm, aligning them with silence‑aware DTW and applying zero‑shot voice conversion to supervise a causal accent translator. The system achieves an 81% reduction in non‑native accent confidence, improves accentedness ratings, reduces speaker linkability in embedding space, and operates with ≤241 ms end‑to‑end latency on a single GPU.