The paper introduces a scalable method for creating pseudo‑triplets—(reference utterance, direction text, modified utterance)—to train direction‑following text‑to‑speech systems. By using an impression‑controllable TTS model to generate style variations and a large language model to craft natural language directions from impression differences, the approach overcomes the scarcity of real training data. Experiments show that these pseudo‑triplets alone yield stable speaker‑preserving modifications, and combining them with recorded data further enhances direction alignment while maintaining speaker similarity.
arXiv:2608. 11590v1 Announce Type: cross Abstract: Human voice generation has made rapid progress in speech generation, singing voice generation, voice cloning, and voice editing.
By Haowei Lou, Hye-Young Paik, Dai Jia, Kai Li, Lina Yao
arXiv:2606. 19325v1 Announce Type: cross Abstract: Existing multi-speaker dialogue systems bind speakers to utterances through structured supervision: per-turn tags, multi-stream transcriptions, or learnable speaker embeddings.
By Michael Finkelson, Daniel Segal, Eitan Richardson, Shahar Armon, Nani Goldring, Poriya Panet, Nir Zabari, Benjamin Brazowski, Or Patashnik, Yoav HaCohen
The paper introduces Speech-Rewarded Style Planning (SRSP), a method that trains a text-based style planner using a frozen downstream TTS model. SRSP generates candidate style instructions from dialogue history and response text, optimizing them with group-relative policy optimization (GRPO) where the reward is the teacher‑forced likelihood of target speech tokens. Experiments on an English subset of the ISCSLP 2026 CoT‑TTS corpus show that SRSP outperforms baseline LLM and target‑audio‑informed captioning approaches in speech‑style and emotion similarity, mel‑cepstral distortion, and contextual appropriateness.
"whyItMatters":"The study demonstrates a more effective way to align natural‑language style descriptions with actual acoustic output, improving controllability and fidelity in conversational text‑to‑speech systems."
By Shiao Zhu, Lianbo Liu, Sizhen Lyu, Yuzhe Wang, Sheng Li, Takahiro Shinozaki
arXiv:2603. 04219v2 Announce Type: replace-cross Abstract: We investigate the use of zero-shot text-to-speech (ZS-TTS) as a data augmentation source for low-resource personalized speech synthesis.
By Youngwon Choi, Jinwoo Oh, Hwayeon Kim, Hyeonyu Kim
Existing multi-speaker dialogue systems bind speakers to utterances through structured supervision: per-turn tags, multi-stream transcriptions, or learnable speaker embeddings. These systems operate within speech-only pipelines that produce clean vocal sequences without the ambient texture of real conversations.