arXiv AI
Jul 9
Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs
arXiv:2607. 06831v1 Announce Type: cross Abstract: Speech-to-text alignment means finding the temporal boundaries of each word in the audio.
OpenPA: Forced-alignment-free Multi-granular Pronunciation Assessment with Mixed-transcript Adaptation
OpenPA is a new automatic pronunciation assessment framework that eliminates the need for forced alignment by using Whisper’s cross‑attention to predict word‑ and utterance‑level scores. It adapts to mixed transcripts through mixed‑transcript adaptation and adaptive substitution weighting, enabling a single model to handle both closed‑ and open‑response settings. Trained only on speechocean762, OpenPA outperforms other systems on in‑domain closed and simulated open‑response tests, retains strong zero‑shot performance on MultiPA, and shows promise in oral‑cancer speech evaluation.
Related stories
arXiv Computation and Language
Sep 22
Closing the Speech-Text Gap with Limited Audio for Effective Domain Adaptation in LLM-Based ASR
arXiv:2604.06487v2 Announce Type: replace Abstract: Conventional end-to-end automatic speech recognition (ASR) systems rely on paired speech-text data for domain adaptation. Recent LLM-based ASR arch...
arXiv Computation and Language
Sep 17
Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs
arXiv:2609.17981v1 Announce Type: cross Abstract: Speech Large Language Models (Speech-LLMs), typically built from a pre-trained speech encoder, a modality projector, and an LLM fine-tuned with Low-R...
arXiv AI
Sep 10
CTC-TTS: LLM-based dual-streaming text-to-speech with CTC alignment
arXiv:2602.19574v2 Announce Type: replace-cross Abstract: Large-language-model (LLM)-based text-to-speech (TTS) systems can generate natural speech, but most are not designed for low-latency dual-str...
arXiv AI
Oct 16
Adaptive Duration Model for Text Speech Alignment
The paper introduces an Adaptive Duration Model designed to predict phoneme-level duration distributions directly from text, improving alignment in text-to-speech systems. It outperforms existing baseline models in precision and adaptability to varying conditions, enhancing phoneme-level alignment accuracy. The model also strengthens zero‑shot TTS performance by reducing sensitivity to mismatches between prompt and input audio.
arXiv Machine Learning
Sep 23
Text-only adaptation in LLM-based ASR through text denoising
arXiv:2601.20900v4 Announce Type: replace-cross Abstract: Adapting large language model (LLM)-based automatic speech recognition (ASR) systems to new domains using text-only data is a significant yet...
arXiv Computation and Language
Aug 25
Mitigating Speaker Leakage in Cascaded Multi-talker ASR with Diarization-based Transcript Correction
arXiv:2608.22196v1 Announce Type: cross Abstract: While cascaded multi-talker ASR (MT-ASR) leverages state-of-the-art foundation models, its performance is often capped by speaker leakage during sepa...
Hugging Face Trending Papers
Sep 16
Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs
Speech Large Language Models (Speech-LLMs), typically built from a pre-trained speech encoder, a modality projector, and an LLM fine-tuned with Low-Rank Adapters (LoRA), have shown strong Automatic Sp...
arXiv Computation and Language
Sep 22
Rethinking Speech-LLM Integration for ASR: Effective Joint Speech-Text Training by Interleaving
arXiv:2607.01733v2 Announce Type: replace Abstract: Speech-LLM integration has shown promising results by leveraging extensive textual pretraining, yet its specific benefits for automatic speech reco...
arXiv AI
Jun 2
PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects
arXiv:2606. 01016v1 Announce Type: cross Abstract: While End-to-End (E2E) Speech-Large Language Models (Speech-LLMs) are rapidly evolving, their evaluation methodologies remain limited to the era of simple transcription.
arXiv AI
Aug 20
Aslema at NADI 2026: Augmentation through Fewshot for SLU
Aslema is a system developed for the NADI 2026 Shared Task 5, which involves intent recognition and slot filling. The team evaluated four omni LLMs in a zero‑shot setting and found that fine‑tuned models consistently outperform zero‑shot inference. They further improved performance by augmenting data with culturally grounded Tunisian Derja utterances generated by an LLM and synthetic speech produced via voice cloning, achieving top‑ranked results on the official test set.
arXiv AI
Oct 26
MG-REPA: Multi-Granularity Representation Alignment for Generative Target Speaker Extraction
MG-REPA introduces Multi-Granularity Representation Alignment for generative target speaker extraction, aligning hidden states with a self‑supervised learning model at the frame level and a speaker verification model at the utterance level. Temporal normalization removes utterance‑level statistics from the frame‑level target, allowing each branch to focus on its specific granularity. On Libri2Mix, MG-REPA achieves comparable or better performance to a 2,000‑epoch baseline while requiring only 750 epochs, and ablation studies confirm its superiority over single‑granularity alignment.