arXiv:2609.09719v1 Announce Type: new Abstract: Text-aligned speech tokenization methods have emerged to better align speech tokens with LLM token spaces, enabling more effective utilization of pretr...
Text-aligned speech tokenization methods have emerged to better align speech tokens with LLM token spaces, enabling more effective utilization of pretrained LLMs. However, they rely on offline automat...
Uni-Stream is a unified streaming speech language model that supports both speech recognition and synthesis within a single architecture. Built on compact pretrained speech models, it processes interleaved text and speech in real time and achieves performance comparable to task‑specific models through a two‑stage training approach. The authors also introduce a data annotation pipeline for streaming training and plan to release the generated transcriptions and word‑level timestamps, while analyzing how speech and text representations occupy distinct latent spaces in the model.
GSLM2 is a multi‑stream generative spoken language model that learns complementary speech representations at multiple abstraction levels using the SpidR 2.0 encoder and a shared‑backbone SLM with stream‑specific predictors. It improves linguistic competence and scales more efficiently than prior speech models, narrowing the performance gap with text‑based language models by about two orders of magnitude on several benchmarks. The authors release the training code and checkpoints publicly.
The paper introduces a new target‑speaker unlearning task for automatic speech recognition (TSU‑ASR) that allows certain speakers to opt out of transcription while still indicating their presence. A lightweight Enrollment‑Conditioned Gating (ECG) module is added to a frozen dual‑stream speech LLM, enabling dynamic unlearning of new opt‑out speakers during inference. Experiments on AMI and AliMeeting datasets show significant drops in transcription accuracy for opt‑out speakers while preserving performance for retained speakers.
Open Thai automatic speech recognition (ASR) is dominated by offline, Whisper-based models that read the whole utterance before transcribing, ruling out low-latency uses such as live captioning and vo...
The paper presents a data‑centric approach to improve automatic speech recognition for non‑verbal vocalizations (NVVs) in the ISCSLP NVVSpeech Challenge. It introduces cross‑dataset label harmonization and a two‑stage sampling schedule—first square‑root category sampling to address long‑tailed distributions, then uniform‑category fine‑tuning—to jointly transcribe lexical content and 16 NVV categories. The final system achieved an official score of 63.86, ranking fourth in Track 1.
arXiv:2609.14991v1 Announce Type: new Abstract: Open Thai automatic speech recognition (ASR) is dominated by offline, Whisper-based models that read the whole utterance before transcribing, ruling ou...
arXiv:2609.23462v1 Announce Type: cross Abstract: Non-verbal vocalizations (NVVs) carry important paralinguistic information but are often omitted by conventional automatic speech recognition (ASR) s...
The paper introduces AWED, a word‑level emission‑delay metric, and demonstrates that post‑training a delayed‑streaming model with a joint reward (GRPO) improves both accuracy and latency. Using a single operating point (τ=6 frames), the method outperforms both its supervised baseline and the Voxtral Realtime backbone across all lookahead budgets, reducing WER by up to 30.8% at 80 ms delay and lowering median AWED from 1.17 s to 1.04 s.
arXiv:2607. 08111v1 Announce Type: cross Abstract: Training target speaker extraction (TSE) models for real conversational mixtures remains challenging because large-scale training corpora and clean target speech for supervision are unavailable.
arXiv:2606. 19325v1 Announce Type: cross Abstract: Existing multi-speaker dialogue systems bind speakers to utterances through structured supervision: per-turn tags, multi-stream transcriptions, or learnable speaker embeddings.