arXiv:2609.09719v1 Announce Type: new Abstract: Text-aligned speech tokenization methods have emerged to better align speech tokens with LLM token spaces, enabling more effective utilization of pretr...
Text-aligned speech tokenization methods have emerged to better align speech tokens with LLM token spaces, enabling more effective utilization of pretrained LLMs. However, they rely on offline automat...
The paper introduces Alignment-Free Text‑Audiobox (Text‑AB), a unified diffusion‑based framework that performs high‑quality voice dubbing and full‑duplex dialogue synthesis without requiring forced alignment. Text‑AB uses a latent diffusion model with DAC‑VAE features, achieving over 10× compression compared to prior EnCodec representations, and learns text‑speech alignment via cross‑attention. The authors pretrain a 3B‑parameter model on 480k hours of monolingual speech and fine‑tune it for cross‑lingual dubbing, full‑duplex dialogue, and emotional dialogue, reporting significant improvements in prosody, voice similarity, naturalness, and emotional expressivity over existing internal systems.
Uni-Stream is a unified streaming speech language model that supports both speech recognition and synthesis within a single architecture. Built on compact pretrained speech models, it processes interleaved text and speech in real time and achieves performance comparable to task‑specific models through a two‑stage training approach. The authors also introduce a data annotation pipeline for streaming training and plan to release the generated transcriptions and word‑level timestamps, while analyzing how speech and text representations occupy distinct latent spaces in the model.
arXiv:2607.01733v2 Announce Type: replace Abstract: Speech-LLM integration has shown promising results by leveraging extensive textual pretraining, yet its specific benefits for automatic speech reco...
arXiv:2608. 08638v1 Announce Type: cross Abstract: Zero-shot text-to-speech (TTS) now supports interactive assistants, personalized media, and accessibility tools.
arXiv:2607. 06831v1 Announce Type: cross Abstract: Speech-to-text alignment means finding the temporal boundaries of each word in the audio.
arXiv:2606. 09048v1 Announce Type: cross Abstract: Removing intermediate representations and separately trained decoding stages has become an important direction in generative modeling.
arXiv:2605. 22083v2 Announce Type: replace-cross Abstract: While flow-matching text-to-speech (TTS) achieves strong zero-shot speaker similarity and naturalness, it remains susceptible to content fidelity issues, particularly skip and repeat errors from imperfect alignment.
The paper introduces an Adaptive Duration Model designed to predict phoneme-level duration distributions directly from text, improving alignment in text-to-speech systems. It outperforms existing baseline models in precision and adaptability to varying conditions, enhancing phoneme-level alignment accuracy. The model also strengthens zero‑shot TTS performance by reducing sensitivity to mismatches between prompt and input audio.
arXiv:2609.18516v1 Announce Type: new Abstract: While Large Language Models excel in natural language processing, efficiently extending their capabilities to spoken input remains a significant challe...
arXiv:2607. 21332v1 Announce Type: cross Abstract: Phonetic forced alignment is a key technique in phonetic research, yet existing alignment systems lack specialized models for low-resource language varieties.