PERSIST is a persistent memory system designed for multi-session, multi-speaker spoken dialogue, explicitly modeling Who, What, and When to enable voice assistants to recall past conversations and adapt to individual users. It structures cross-session histories into readable event records and retrieves them using a 3W joint scoring mechanism that combines semantic content, acoustic speaker identity, and temporal state. By reusing intermediate representations from the dialogue backbone, PERSIST reduces retrieval latency from 578.42 ms to 7.03 ms and achieves 85.08% end‑to‑end task accuracy on the SpokenTrace benchmark, improving EM@3 from 49.01% to 82.10%.
By Achira Lin, Siyuan Hou, Wenyi Yu, Xinnian Zhao, Haoyu Niu, Wang Geng, Longshuai Xiao, Shihai Xiao, Mangsuo Zhao, Chao Zhang
arXiv:2609.39344v1 Announce Type: cross
Abstract: Speech transcripts used as long-term memory must preserve both words and stable speaker identities. Existing meeting-transcription metrics either ign...
By Shantanu Vispute, Aditya Mishra, Siddhartha Saxena
Speech transcripts used as long-term memory must preserve both words and stable speaker identities. Existing meeting-transcription metrics either ignore speakers or remap anonymous speakers independen...
The paper introduces a black-box membership inference attack framework tailored for fine-tuned text-to-speech models, addressing challenges in query generation and representation engineering. It evaluates five query types, finding recitation queries most effective, and uses multi-level speech embeddings with temporal alignment for fine-grained comparison. Experiments on CosyVoice2, F5-TTS, and XTTS-v2 trained on VCTK and British Dialect datasets show high privacy leakage, with speaker-level AUC above 0.80 and record-level AUC between 0.80 and 0.90.
By Kunlin Cai, Kaiyuan Zhang, Zihang Xiang, Jinghuai Zhang, Abeer Alwan, Fnu Suya, Yuan Tian
The paper introduces a new target‑speaker unlearning task for automatic speech recognition (TSU‑ASR) that allows certain speakers to opt out of transcription while still indicating their presence. A lightweight Enrollment‑Conditioned Gating (ECG) module is added to a frozen dual‑stream speech LLM, enabling dynamic unlearning of new opt‑out speakers during inference. Experiments on AMI and AliMeeting datasets show significant drops in transcription accuracy for opt‑out speakers while preserving performance for retained speakers.
By Bo Su, Yueru Yan, Thai Le
The paper introduces GUARD, a lightweight speaker identity unlearning framework designed to prevent re-identification in zero-shot text-to-speech systems. GUARD employs a learned speaker gate and speaker-agnostic activation steering on a frozen TTS backbone, optimizing steering vectors through group-relative reward optimization to reduce similarity to forgotten speakers while maintaining intelligibility and naturalness. Experiments on CosyVoice2 show that GUARD significantly lowers forget-speaker similarity and re-identification accuracy while preserving the ability to reproduce retained speakers.
By Hyoeun Kim, Yujun Lee, Kyuhong Shim
arXiv:2610.08828v1 Announce Type: new
Abstract: Small-data adaptation can improve speech detection while degrading speaker attribution. We study this discrepancy in a released streaming diarizer adap...
By Mo Yu, Yang Liu, Jing Qian
arXiv:2609.26780v1 Announce Type: cross
Abstract: Long-term conversational memory in multi-party settings requires more than retrieving relevant content from long-term conversations: it must distingu...
By Haobo Zheng, Tan Tang, Yan Chen, Weijie Wang, Yingcai Wu
arXiv:2609.20995v1 Announce Type: cross
Abstract: Natural spoken interaction requires more than streaming ASR, language generation, and speech synthesis: a system must react to overlap without cancel...
By Bertil Braun
arXiv:2603.07551v3 Announce Type: replace-cross
Abstract: Recent zero-shot Text-to-Speech (TTS) systems can clone previously unseen voices from only a few seconds of audio. We formulate Speech Genera...
By Thanathai Lertpetchpun, Thanapat Trachu, Sai Praneeth Karimireddy, Shrikanth Narayanan
MENASpeechBank is a new reference voice bank that provides about 18,000 high‑quality utterances from 124 speakers across multiple MENA countries, covering English, Modern Standard Arabic, and regional Arabic varieties. The dataset is built through a controllable pipeline that creates persona profiles inspired by the World Values Survey, defines a taxonomy of roughly 5,000 conversational scenarios, matches personas to scenarios via semantic similarity, and generates around 417,000 role‑play conversations using an LLM. Synthetic speaker‑conditioned user‑turn audio is produced from reference recordings to maintain speaker diversity, and both synthetic and human‑recorded conversations are evaluated and analyzed for quality.
By Zien Sheikh Ali, Hunzalah Hassan Bhatti, Rabindra Nath Nandi, Shammur Absar Chowdhury, Firoj Alam
arXiv:2607. 21820v1 Announce Type: cross Abstract: Audio deepfake detectors are trained to distinguish genuine speech from synthetic speech and often perform well on standard benchmarks.
By Daniyal Kabir Dar, Arun Ross