arXiv Computation and Language

Controllable Dysarthric Speech Synthesis with Patient-Specific Conditioning for Speaker-Diverse ASR Augmentation

arXiv AI
Aug 20

When Audio-Language Models Fail to Leverage Multimodal Context for Dysarthric Speech Recognition

The paper evaluates whether audio‑language models can use multimodal clinical context to improve dysarthric speech recognition. Using a benchmark built on the Speech Accessibility Project dataset, the authors test diagnosis labels, clinician ratings, and detailed clinical descriptions as prompts for nine models. They find that these prompts yield negligible or negative effects on word error rate, though fine‑tuning with LoRA and mixed prompt formats reduces WER by 52% and benefits certain subgroups such as Down syndrome and mild‑severity speakers.

By Pehu\'en Moure, Niclas Pokel, Bilal Bounajma, Yingqiang Gao, Roman Boehringer, Longbiao Cheng, Shih-Chii Liu
Hugging Face Trending Papers
Jul 20

Re-Sonance: A Dysarthric Asynchronous Real-Time Speech Conversion System Based on a Three-Stage Cascaded ASR-LLM-TTS Architecture

Individuals with dysarthria face significant challenges in professional speaking scenarios such as conferences, presentations, and meetings, where real-time communication is crucial. While existing Augmentative and Alternative Communication (AAC) systems provide basic support, they often fail to meet the demands of professional speaking environments due to high latency and unnatural speech patterns.

arXiv Machine Learning
Oct 9

Self-Supervised Speech Representations for Cross-Speaker Dysarthria Detection During Awake Craniotomy

The study evaluates a pipeline for detecting dysarthria during awake craniotomy using the DATABRASE corpus. It combines speaker diarization, a multi‑view representation of handcrafted acoustic features and wav2vec 2.0 embeddings, speaker‑conditional normalization, feature selection, and a cascaded classifier. Results show that the speech representation has a larger impact on cross‑speaker performance than classifier choice, with self‑supervised embeddings improving AUC by 18.2%–26.1% and additional gains from diarization‑conditioned extraction and the classifier cascade.

By Kanthila Chinmayi (IRDL, LaTIM), Abdallah Nassib (LARIS), Misy Harrison (LaTIM), Panheleux Celine (LaTIM, CHU - BREST), Saliou Vanessa (CHU - BREST), Seizeur Romuald (LaTIM), Dardenne Guillaume (LaTIM)
arXiv AI
Jun 19

Cross-Dataset, Age, and Gender Generalization: A Comprehensive Analysis of Fine-Tuning Strategies for Low-Resource Children's ASR

arXiv:2606. 19791v1 Announce Type: cross Abstract: The challenge associated with recognizing dysarthric speech primarily arises from pronounced acoustic variability attributed to impaired articulatory precision.

By Paban Sapkota, Hemant Kumar Kathania, Mikko Kurimo, Sudarsana Reddy Kadiri, Shrikanth Narayanan
arXiv AI
Oct 5

Personalized Automatic Speech Recognition for a Dysarthric and Tracheostomic Speaker using Artificial Conversations

This paper introduces a personalized automatic speech recognition system for a Czech speaker with a permanent tracheal stoma and severe dysarthria. The authors release a 33‑hour annotated dataset collected via an artificial conversation protocol and develop a multi‑stage training pipeline based on Whisper Base, fine‑tuning on Czech speech, simulated tracheostomic speech, and the speaker’s data. Evaluations in scripted, question‑answering, and spontaneous dialogue scenarios show a 50 % relative reduction in character error rate compared to the Whisper Base baseline and better accuracy than the speaker’s assistants on isolated utterances.

By David Nadrchal, Monorama Swain, Florian Schmid, Gerhard Widmer, Paul Primus