The paper investigates how to optimize non‑verbal vocalizations (NVs) such as laughter, coughs, and sighs in text‑to‑speech systems. It introduces a preference‑optimization framework that uses preference signals, pair construction, and DPO objectives, and defines an NV‑aware character error rate (NV‑CER) to guide controllable NV generation. Experiments on Emilia‑NV and NV‑Bench demonstrate that the proposed setup improves NV realization and lexical fidelity, with objective, LLM‑based, and human evaluations supporting the results.
Non-verbal vocalizations (NVs), such as laughter, coughs, and sighs, are essential for expressive TTS, but the effectiveness of preference optimization for NV generation remains poorly understood. We...
arXiv:2609.14231v1 Announce Type: cross Abstract: Controllable synthesis of nonverbal vocalizations (NVVs) is es- sential for natural and expressive speech, but remains challeng- ing due to their aco...
arXiv:2602. 03420v2 Announce Type: replace-cross Abstract: Emotional expression in human speech is nuanced and compositional, often involving multiple, sometimes conflicting, affective cues that may diverge from linguistic content.
arXiv:2609.01246v1 Announce Type: new Abstract: Current Large Language Models (LLMs) are primarily optimized for written text, often producing outputs that are grammatically correct and helpful yet p...
arXiv:2606. 31729v1 Announce Type: cross Abstract: Text-to-speech (TTS) evaluation is an open challenge.
Chinese news text contains dense written forms such as scores, hyphenated model names, ranges, unit symbols, percentages, English abbreviations, and mixed Chinese-Latin-digit names. These forms are frequent in real listening workflows, and a text-to-speech (TTS) system can preserve the written string while changing the spoken meaning.
The paper introduces Live-ProsodyJudge (LPJ), a cost‑effective pairwise evaluator distilled from Gemini for assessing fine‑grained prosody in live streaming TTS. It identifies a flaw called verdict coupling, where multi‑dimensional scores collapse into a single preference, and proposes Decoupled‑Live‑ProsodyJudge (D‑LPJ) to eliminate this issue through masking and a span‑local GRPO strategy. Experiments show LPJ outperforms a single Gemini call in accuracy, and D‑LPJ provides independent dimension judgments, achieving high alignment with human top‑3 selections in a TTS candidate tournament.
arXiv:2606. 15888v1 Announce Type: cross Abstract: Non-verbal vocalizations (NVs), such as laughter, sighs, and coughs, are important acoustic cues for emotion and intent.
EmphTTS is a non‑autoregressive text‑to‑speech system that uses Group Relative Policy Optimization (GRPO) to train its duration predictor with an emphasis localization reward, allowing direct word‑level emphasis control. The system outperforms existing baselines in both objective emphasis evaluation and subjective preference tests, and ablation studies confirm that GRPO improves emphasis realization while reducing mismatch between the duration predictor and the TTS model.
arXiv:2601.09050v2 Announce Type: replace Abstract: Tonal low-resource languages are widely spoken but remain underserved by modern speech technologies. A central challenge is learning speech represe...
The paper introduces a unified phoneme‑based TTS‑to‑ASR augmentation pipeline that uses a multilingual TTS model with language‑ID conditioning and incorporates grapheme‑to‑phoneme conversion, reference‑speech filtering, and candidate‑text selection. It proposes phoneme‑frequency‑guided selection (PFGS) to rank sentences based on phoneme frequencies from real ASR labels, and demonstrates that random augmentation and PFGS both improve ASR performance across Arabic, French, Italian, and Portuguese test sets, with PFGS yielding up to a 19.3% relative WER reduction. The study also shows that filtering reference speech can further lower WER by up to 0.59 points on certain datasets.
arXiv:2608. 11590v1 Announce Type: cross Abstract: Human voice generation has made rapid progress in speech generation, singing voice generation, voice cloning, and voice editing.