The paper presents Conversational Voice Aesthetic Model (CVAM), a speech large language model that describes voice aesthetics in conversational contexts. CVAM predicts nine categorical attributes such as gender, pitch, pacing, emotion, and delivery, and is trained on human-annotated data from the CANDOR corpus. Experiments show that CVAM aligns better with human listeners than Gemini 3.1 Pro and other open‑source speech LLMs, demonstrating the value of grounding voice aesthetics in human perception.
arXiv:2607. 15755v1 Announce Type: cross Abstract: Conversational Speech Synthesis (CSS) aims to synthesize speech with human-like emotional expression and contextual consistency in user-agent interactions.
arXiv:2606. 31729v1 Announce Type: cross Abstract: Text-to-speech (TTS) evaluation is an open challenge.
The paper introduces an LLM-based framework for continuous dimensional emotion evaluation in multimodal dialogue, combining discrete emotion recognition with Valence-Arousal-Dominance (VAD) assessment on the IEMOCAP dataset. It incorporates acoustic cues as natural language descriptions via the SpeechCueLLM approach and evaluates six models from the LLaMA, GPT, and Qwen families using zero-shot, few-shot, and LoRA fine-tuning. LoRA-fine-tuned LLaMA models outperform prompt-engineered GPT models, achieving a new state-of-the-art Valence CCC of 0.7822, and ablation studies show that textual audio descriptions significantly benefit smaller models. "whyItMatters":"The study demonstrates that domain adaptation through fine-tuning can surpass larger GPT models in multimodal emotion evaluation, highlighting the importance of tailored training for emotion recognition tasks."
arXiv:2608. 09930v1 Announce Type: cross Abstract: Automated Text-to-Speech (TTS) evaluation methods (Mean Opinion Score (MOS) predictors and Audio Large Language Models (Audio-LLM) judges) are expected to reflect human perception, yet it is unclear how well they capture the distinct aspects of speech that listeners actually perceive.
arXiv:2601. 02813v3 Announce Type: replace Abstract: Aligning language models to qualitative behavioral traits, such as human-likeness, remains difficult because they are hard to define, measure, and optimize.
arXiv:2609.10394v1 Announce Type: cross Abstract: Current audio-visual speech recognition (AVSR) benchmarks, like LRS3, rely heavily on clean, scripted and rehearsed speech. They fail to reflect the...
The paper reports the first Turing test for speech‑to‑speech systems, gathering 2,968 human judgments on conversations between nine state‑of‑the‑art S2S systems and 28 humans. None of the evaluated systems passed the test, highlighting a clear gap in human‑likeness. The authors diagnose the failure with an 18‑dimension taxonomy, finding that paralinguistic cues, emotional expressivity, and conversational persona—not semantic understanding—are the main bottlenecks, and they propose an interpretable model for automatic human‑vs‑machine discrimination.
arXiv:2510.22588v2 Announce Type: replace-cross Abstract: Spoken dialogue models currently lack the ability for fine-grained speech style control, a critical capability for human-like interaction tha...
The paper introduces Speech-Rewarded Style Planning (SRSP), a method that trains a text-based style planner using a frozen downstream TTS model. SRSP generates candidate style instructions from dialogue history and response text, optimizing them with group-relative policy optimization (GRPO) where the reward is the teacher‑forced likelihood of target speech tokens. Experiments on an English subset of the ISCSLP 2026 CoT‑TTS corpus show that SRSP outperforms baseline LLM and target‑audio‑informed captioning approaches in speech‑style and emotion similarity, mel‑cepstral distortion, and contextual appropriateness. "whyItMatters":"The study demonstrates a more effective way to align natural‑language style descriptions with actual acoustic output, improving controllability and fidelity in conversational text‑to‑speech systems."
The paper introduces ReACT‑TTS, a two‑stage framework that uses a one‑second sequence of listener facial reactions to plan the emotion and prosody of a conversational utterance before speech synthesis. Experiments on the MELD dataset show that temporally conditioned models outperform text‑only baselines in macro‑F1 and VAD concordance, while accuracy remains unchanged. A contextual‑appropriateness study with speech researchers indicates a strong preference for temporally conditioned responses, and the approach is integrated with a Grad‑TTS backbone for end‑to‑end speech generation.
arXiv:2606. 20400v1 Announce Type: new Abstract: Generating high-utility synthetic data for intent classification typically requires human-annotated seed data, which is often unavailable in fast-paced industrial settings.
arXiv:2606. 14922v1 Announce Type: cross Abstract: For the last couple of years, the field of speech synthesis has improved dramatically thanks to deep learning.