← Back to all news
arXiv Machine Learning By Denis Pavlov, Ulanbek Abdurazakov, Nursultan Bakashov

GEPARD - Generative, Prosody-aware, Autoregressive text-to-speech model for Realtime Dialogue

Read the original on arXiv Machine Learning →

GEPARD is a streaming text‑to‑speech model that uses a standard large language model backbone to generate speech autoregressively, decoding audio with an FSQ‑based neural codec. It streams audio chunk‑by‑chunk as text arrives, achieving a real‑time factor of about 0.067 and an aggregate speedup of roughly 204× on a single GPU with 256 concurrent streams. The design keeps all complex auxiliary mechanisms outside the decode loop, enabling deployment with a standard LLM engine (vLLM) without kernel modifications.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 17

Look Less, Hear Better: Jointly Rewarded GRPO for Streaming ASR

The paper introduces AWED, a word‑level emission‑delay metric, and demonstrates that post‑training a delayed‑streaming model with a joint reward (GRPO) improves both accuracy and latency. Using a single operating point (τ=6 frames), the method outperforms both its supervised baseline and the Voxtral Realtime backbone across all lookahead budgets, reducing WER by up to 30.8% at 80 ms delay and lowering median AWED from 1.17 s to 1.04 s.

By Xiuwen Zheng
llmsfine-tuningmultimodalsafety
More like this →
arXiv Machine Learning
Sep 10

TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context

TontaubeV1 is a streaming text‑to‑speech model that preserves natural prosody while running on a single consumer GPU. It encodes speech with a hierarchical DualCodec representation at 12.5 Hz, separating a semantic stream from successive acoustic refinements, and uses Qwen3‑derived transformers to predict the semantic stream, utterance duration, and acoustic refinements. The system supports up to one minute of reference audio for voice conditioning, streams with a 200 ms latency to first audio, and achieves real‑time factors of 0.08 (single input) and 0.02 (eight concurrent inputs).

By Fritz Cremer, Jonathan Cremer
llmsdiffusionmultimodalbenchmarks
More like this →
arXiv AI
Jun 9

Liberating LLM Capabilities in Full-Duplex Speech Models

arXiv:2606. 07547v1 Announce Type: cross Abstract: Speech-based large language models are typically constrained to spoken replies, which limits their user-facing outputs to what can be verbalized and suppresses text-native capabilities such as code generation, structured analysis, and multi-step reasoning in realtime interaction, for tasks that require persistent, structured, and inspectable intermediate outputs.

By Luoyuan Zhang, Bokai Xu, Junbo Cui, Weiyue Sun, Yingjing Xu, Hanyu Liu, Yuan Yao
llmsdata-engineering
More like this →
Hugging Face Trending Papers
Sep 8

TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context

TontaubeV1 is a streaming text‑to‑speech model that maintains natural prosody while running on a single consumer GPU. It encodes speech with a hierarchical DualCodec representation at 12.5 Hz, separating a semantic stream from successive acoustic refinements, and uses Qwen3‑derived transformers to predict the semantic stream and add refinements. The model supports up to one minute of reference audio for voice conditioning, streams with a 200 ms latency to first audio, and achieves real‑time factors of 0.08 (single input) and 0.02 (eight concurrent inputs).

llmsdiffusionmultimodalbenchmarks
More like this →