← Back to all news
arXiv AI By Pehu\'en Moure, Niclas Pokel, Bilal Bounajma, Yingqiang Gao, Roman Boehringer, Longbiao Cheng, Shih-Chii Liu

When Audio-Language Models Fail to Leverage Multimodal Context for Dysarthric Speech Recognition

Read the original on arXiv AI →

The paper evaluates whether audio‑language models can use multimodal clinical context to improve dysarthric speech recognition. Using a benchmark built on the Speech Accessibility Project dataset, the authors test diagnosis labels, clinician ratings, and detailed clinical descriptions as prompts for nine models. They find that these prompts yield negligible or negative effects on word error rate, though fine‑tuning with LoRA and mixed prompt formats reduces WER by 52% and benefits certain subgroups such as Down syndrome and mild‑severity speakers.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computation and Language
Sep 15

Controllable Dysarthric Speech Synthesis with Patient-Specific Conditioning for Speaker-Diverse ASR Augmentation

arXiv:2602.08696v3 Announce Type: replace-cross Abstract: Dysarthric speech recognition is limited by high speaker variability and scarce labeled data. Existing synthesis methods often couple speaker...

By Haoshen Wang, Xueli Zhong, Bingbing Lin, Jia Huang, Xingduo Pan, Shengxiang Liang, Nizhuan Wang, Wai Ting Siok
llmsfine-tuningmultimodal
More like this →
arXiv Machine Learning
Oct 9

Self-Supervised Speech Representations for Cross-Speaker Dysarthria Detection During Awake Craniotomy

The study evaluates a pipeline for detecting dysarthria during awake craniotomy using the DATABRASE corpus. It combines speaker diarization, a multi‑view representation of handcrafted acoustic features and wav2vec 2.0 embeddings, speaker‑conditional normalization, feature selection, and a cascaded classifier. Results show that the speech representation has a larger impact on cross‑speaker performance than classifier choice, with self‑supervised embeddings improving AUC by 18.2%–26.1% and additional gains from diarization‑conditioned extraction and the classifier cascade.

By Kanthila Chinmayi (IRDL, LaTIM), Abdallah Nassib (LARIS), Misy Harrison (LaTIM), Panheleux Celine (LaTIM, CHU - BREST), Saliou Vanessa (CHU - BREST), Seizeur Romuald (LaTIM), Dardenne Guillaume (LaTIM)
rag
More like this →