The study investigates how intonation is encoded in text‑to‑speech models that use residual vector quantization. By probing different components of the architecture, the authors find that the transformer backbone’s hidden state carries the nuclear intonation pattern, while the first token level does not. Steering this backbone state with vectors derived from human‑imitated intonation patterns can shift the generated speech toward falling or rising melodies, though precise control remains limited and naturalness slightly decreases.
arXiv:2606. 05852v1 Announce Type: cross Abstract: Text-to-speech (TTS) and singing voice synthesis (SVS) both aim to generate human vocal audio from symbolic inputs, but they impose different requirements on the generation process.
arXiv:2606. 11400v1 Announce Type: cross Abstract: Large Audio-Language Models (LALMs) excel at audio understanding but expose little about where in an audio signal they attend.
arXiv:2607. 25270v1 Announce Type: cross Abstract: Activation steering controls language models by adding vectors or features to hidden states at inference time, but the upstream source of these steering signals is often treated as a secondary detail.
arXiv:2608.28970v1 Announce Type: cross Abstract: Current TTS systems typically rely on open-loop, single-pass generation and can produce sporadic local prosodic defects, such as misplaced stress, un...
Prosodic cues can convey task-relevant information that alters the trajectory and outcome of a task-oriented dialogue, even when the words themselves remain unchanged. Yet existing benchmarks typically evaluate prosodic perception, response appropriateness, and task-oriented dialogue in isolation, making it difficult to test whether prosodic evidence changes downstream decisions.
arXiv:2608. 19515v1 Announce Type: new Abstract: Prosodic cues can convey task-relevant information that alters the trajectory and outcome of a task-oriented dialogue, even when the words themselves remain unchanged.
arXiv:2609.15313v1 Announce Type: cross Abstract: Autoregressive generation of interleaved text and acoustic tokens is a common approach to spoken-response generation in speech large language models....
The paper demonstrates that high linear decodability in continuous‑latent TTS does not guarantee causal controllability of speaker identity. In VoxCPM2, probes at the final residual acoustic language model layer show strong accuracy, yet aligned edits fail to produce target‑dominant dyads, whereas early‑position edits succeed. Cross‑architecture experiments reveal that backbone edits shift the source voice without attracting the target, while x‑vector ablation eliminates source similarity, underscoring the need for geometric specificity and a computational path that embeds the intended speaker condition into synthesis.
arXiv:2606. 11386v1 Announce Type: cross Abstract: Full-duplex spoken language models (FD-SLMs) enable seamless speech interaction by allowing models to listen and speak simultaneously, yet the internal mechanism by which they coordinate listening and speaking remains underexplored.
arXiv:2608. 19211v1 Announce Type: cross Abstract: Human speech is richly expressive, with prosody carrying linguistic and emotional information beyond the lexical content.
arXiv:2609.14231v1 Announce Type: cross Abstract: Controllable synthesis of nonverbal vocalizations (NVVs) is es- sential for natural and expressive speech, but remains challeng- ing due to their aco...
The paper introduces a closed‑loop dual‑reference guidance method for zero‑shot cross‑lingual voice cloning that pairs an original reference with a reusable native anchor created via voice conversion. By initializing separate language‑model states and synchronizing them through shared acoustic trajectories, the approach reduces residual accent leakage while maintaining speaker similarity. Experiments on French speakers show significant reductions in language‑identification mass, improved word‑error and GOP scores, and favorable listener preferences, all achieved with a single‑time anchor construction and lower real‑time factor than a TTS→VC cascade.