The paper introduces a scalable method for creating pseudo‑triplets—(reference utterance, direction text, modified utterance)—to train direction‑following text‑to‑speech systems. By using an impression‑controllable TTS model to generate style variations and a large language model to craft natural language directions from impression differences, the approach overcomes the scarcity of real training data. Experiments show that these pseudo‑triplets alone yield stable speaker‑preserving modifications, and combining them with recorded data further enhances direction alignment while maintaining speaker similarity.
arXiv:2608. 11590v1 Announce Type: cross Abstract: Human voice generation has made rapid progress in speech generation, singing voice generation, voice cloning, and voice editing.
By Haowei Lou, Hye-Young Paik, Dai Jia, Kai Li, Lina Yao
arXiv:2606. 19325v1 Announce Type: cross Abstract: Existing multi-speaker dialogue systems bind speakers to utterances through structured supervision: per-turn tags, multi-stream transcriptions, or learnable speaker embeddings.
By Michael Finkelson, Daniel Segal, Eitan Richardson, Shahar Armon, Nani Goldring, Poriya Panet, Nir Zabari, Benjamin Brazowski, Or Patashnik, Yoav HaCohen
The paper introduces Speech-Rewarded Style Planning (SRSP), a method that trains a text-based style planner using a frozen downstream TTS model. SRSP generates candidate style instructions from dialogue history and response text, optimizing them with group-relative policy optimization (GRPO) where the reward is the teacher‑forced likelihood of target speech tokens. Experiments on an English subset of the ISCSLP 2026 CoT‑TTS corpus show that SRSP outperforms baseline LLM and target‑audio‑informed captioning approaches in speech‑style and emotion similarity, mel‑cepstral distortion, and contextual appropriateness.
"whyItMatters":"The study demonstrates a more effective way to align natural‑language style descriptions with actual acoustic output, improving controllability and fidelity in conversational text‑to‑speech systems."
By Shiao Zhu, Lianbo Liu, Sizhen Lyu, Yuzhe Wang, Sheng Li, Takahiro Shinozaki
arXiv:2603. 04219v2 Announce Type: replace-cross Abstract: We investigate the use of zero-shot text-to-speech (ZS-TTS) as a data augmentation source for low-resource personalized speech synthesis.
By Youngwon Choi, Jinwoo Oh, Hwayeon Kim, Hyeonyu Kim
Existing multi-speaker dialogue systems bind speakers to utterances through structured supervision: per-turn tags, multi-stream transcriptions, or learnable speaker embeddings. These systems operate within speech-only pipelines that produce clean vocal sequences without the ambient texture of real conversations.
arXiv:2608. 13613v1 Announce Type: cross Abstract: Recent breakthroughs in generative models have made text-to-voice generation (TTV) possible, enabling the synthesis of speech directly from textual voice descriptions.
By Jiarui Hai, Karan Thakkar, Ke Chen, Yunyun Wang, Jiaqi Su, Rithesh Kumar, Mounya Elhilali, Zeyu Jin
Poly-InstructTTS is a text‑to‑speech system that learns expressive speech from open‑ended natural‑language instructions using a 1,000‑hour, 1,000‑plus emotion and style annotated audiovisual corpus. The approach employs a prompt‑free GPT with attribute‑based thinking tokens and a flow‑matching module to inject timbre from reference audio, and includes a speaker fine‑tuning procedure to transfer instruction control while preserving speaker persona. Experiments demonstrate strong instruction adherence and expressiveness, with audio demos and an expanded test set available on the project page.
By Junhui Zhang, Qianhui Xu, Qingxiang Guo, Dawei Yang, Ling Miao, Qiangqiang Wang, Yang Song
arXiv:2606. 26534v1 Announce Type: cross Abstract: Recently, zero-shot text-to-speech (TTS) has enabled high-fidelity and expressive speech synthesis, but it often fails to imitate unseen speaking styles from uncommon scenarios (e.
By Tianxin Xie, Chenxing Li, Dong Yu, Li Liu
arXiv:2607. 00363v1 Announce Type: cross Abstract: Flow Matching (FM) has emerged as a powerful paradigm for speech generation but remains constrained by high inference latency and timbre leakage.
By Zuda Yu, Qianhui Xu, Ting Chen, Junhui Zhang, Tao Fu, Hongjiang Yu, Qiangqing Wang, Yang Song
arXiv:2601.12966v2 Announce Type: replace-cross
Abstract: The Lombard effect plays a key role in natural communication, particularly in noisy environments or when addressing hearing-impaired listener...
By Seymanur Akti, Alexander Waibel
arXiv:2608. 02673v1 Announce Type: cross Abstract: Speech editing for content creation requires precise control over both what an edit should do and where it should apply.
By Hankun Wang, Bohan Li, Shi Lian, Xiaoyu Gu, Jing Peng, Da Zheng, Colin Zhang, Kai Yu