The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
arXiv:2610.34623v1 Announce Type: new Abstract: Text injection trains speech recognition models on unpaired text alongside paired audio. We feed the speech encoder synthetic input derived from text s...
arXiv:2607. 06831v1 Announce Type: cross Abstract: Speech-to-text alignment means finding the temporal boundaries of each word in the audio.
Recent advances in zero-shot text-to-speech (TTS) have substantially improved speech quality and voice cloning fidelity. However, many zero-shot TTS systems still depend on audio prompt transcripts at inference time.
The paper introduces TextPro‑SLM, a speech large language model that reduces the modality gap by making spoken input resemble a prosody‑aware text LLM. It combines WhisperPro, a unified speech encoder that outputs synchronized text tokens and prosody embeddings, with an LLM backbone that retains the semantic power of the original text LLM while learning paralinguistic cues. Experiments demonstrate that TextPro‑SLM achieves the lowest modality gap among leading SLMs at both 3B and 7B scales, and performs strongly on paralinguistic understanding tasks using only about 1,000 hours of audio for training.
GSLM2 is a multi‑stream generative spoken language model that learns complementary speech representations at multiple abstraction levels using the SpidR 2.0 encoder and a shared‑backbone SLM with stream‑specific predictors. It improves linguistic competence and scales more efficiently than prior speech models, narrowing the performance gap with text‑based language models by about two orders of magnitude on several benchmarks. The authors release the training code and checkpoints publicly.
arXiv:2607. 13013v1 Announce Type: new Abstract: Automatic speech recognition is dominated by autoregressive decoders that emit one token at a time.