The paper demonstrates that high linear decodability in continuous‑latent TTS does not guarantee causal controllability of speaker identity. In VoxCPM2, probes at the final residual acoustic language model layer show strong accuracy, yet aligned edits fail to produce target‑dominant dyads, whereas early‑position edits succeed. Cross‑architecture experiments reveal that backbone edits shift the source voice without attracting the target, while x‑vector ablation eliminates source similarity, underscoring the need for geometric specificity and a computational path that embeds the intended speaker condition into synthesis.
arXiv:2608. 00722v1 Announce Type: cross Abstract: Language model-based text-to-speech (LM-based TTS) remains vulnerable to speech hallucinations that deviate from the target text.
VoxReason introduces a listener‑free evaluation framework that measures whether a speech planning system’s delivery choices—such as pitch, energy, rate, pause, emphasis, and stance—are grounded in cited source records before any waveform is generated. The system outputs a source‑cited speaking plan and uses a deterministic verifier to check citation legality, slot agreement, unsupported states, schema validity, and counterfactual locality. Experiments on 1,440 source‑label cases show that simple slot accuracy can be misleading, while a 7B locality‑based repair model significantly improves plan‑slot accuracy and locality, and removing source records sharply reduces the grounded score. whyItMatters":"The framework provides a concrete, measurable way to ensure that expressive speech systems make source‑licensed planning decisions, addressing a source‑use failure that occurs before audio synthesis."
arXiv:2606. 01909v1 Announce Type: cross Abstract: We present Echo, a proof-of-concept audio system built around a single 25 M-parameter ViT encoder.
arXiv:2603. 01006v3 Announce Type: replace-cross Abstract: REPresentation Alignment (REPA) improves the training of generative flow models by aligning intermediate hidden states with pretrained teacher features, but its effectiveness in token-conditioned audio Flow Matching critically depends on the choice of supervised layers, which is typically made heuristically based on the depth.
The study investigates how intonation is encoded in text‑to‑speech models that use residual vector quantization. By probing different components of the architecture, the authors find that the transformer backbone’s hidden state carries the nuclear intonation pattern, while the first token level does not. Steering this backbone state with vectors derived from human‑imitated intonation patterns can shift the generated speech toward falling or rising melodies, though precise control remains limited and naturalness slightly decreases.
arXiv:2606. 07080v1 Announce Type: cross Abstract: We present dots.
The paper introduces a method for controlling the turn‑taking behavior of full‑duplex spoken language models by identifying a single late‑layer residual‑stream direction, called the initiative vector, that determines how much the model speaks. By extracting this vector from the model’s own conversations without additional training, the authors can anticipate speech onsets up to 800 ms ahead and adjust the speech ratio from 6–12 % to 42–61 % through scaling during generation. A short reference recording of a target speaker allows the model to adopt that speaker’s turn‑taking style, and the transfer is highly speaker‑specific, with mismatched references degrading performance in 93–98 % of cases.
arXiv:2608. 13817v1 Announce Type: cross Abstract: Human speech production is constrained by physiology, giving rise to characteristic temporal structure on acoustic signals.
arXiv:2609.36577v1 Announce Type: cross Abstract: Audio large language models (ALLMs) can reason about the content of audio recordings to perform complex tasks. However, these capabilities usually co...
arXiv:2609.05871v1 Announce Type: cross Abstract: Audio-conditioned language models often underuse acoustic cues such as prosody, emotion, and non-speech sounds, raising the question of whether ASR-s...
arXiv:2608. 02673v1 Announce Type: cross Abstract: Speech editing for content creation requires precise control over both what an edit should do and where it should apply.
arXiv:2609.15313v1 Announce Type: cross Abstract: Autoregressive generation of interleaved text and acoustic tokens is a common approach to spoken-response generation in speech large language models....