arXiv AI

When Audio-Language Models Fail to Leverage Multimodal Context for Dysarthric Speech Recognition

The paper evaluates whether audio‑language models can use multimodal clinical context to improve dysarthric speech recognition. Using a benchmark built on the Speech Accessibility Project dataset, the authors test diagnosis labels, clinician ratings, and detailed clinical descriptions as prompts for nine models. They find that these prompts yield negligible or negative effects on word error rate, though fine‑tuning with LoRA and mixed prompt formats reduces WER by 52% and benefits certain subgroups such as Down syndrome and mild‑severity speakers.

arXiv Computation and Language
Sep 15

Controllable Dysarthric Speech Synthesis with Patient-Specific Conditioning for Speaker-Diverse ASR Augmentation

arXiv:2602.08696v3 Announce Type: replace-cross Abstract: Dysarthric speech recognition is limited by high speaker variability and scarce labeled data. Existing synthesis methods often couple speaker...

By Haoshen Wang, Xueli Zhong, Bingbing Lin, Jia Huang, Xingduo Pan, Shengxiang Liang, Nizhuan Wang, Wai Ting Siok
arXiv Machine Learning
Oct 9

Self-Supervised Speech Representations for Cross-Speaker Dysarthria Detection During Awake Craniotomy

The study evaluates a pipeline for detecting dysarthria during awake craniotomy using the DATABRASE corpus. It combines speaker diarization, a multi‑view representation of handcrafted acoustic features and wav2vec 2.0 embeddings, speaker‑conditional normalization, feature selection, and a cascaded classifier. Results show that the speech representation has a larger impact on cross‑speaker performance than classifier choice, with self‑supervised embeddings improving AUC by 18.2%–26.1% and additional gains from diarization‑conditioned extraction and the classifier cascade.

By Kanthila Chinmayi (IRDL, LaTIM), Abdallah Nassib (LARIS), Misy Harrison (LaTIM), Panheleux Celine (LaTIM, CHU - BREST), Saliou Vanessa (CHU - BREST), Seizeur Romuald (LaTIM), Dardenne Guillaume (LaTIM)
arXiv Computation and Language
Oct 16

SITAR: A Situational Distortion Metric for Automatic Speech Recognition Evaluation

The paper introduces SITAR, a situational distortion metric for evaluating automatic speech recognition (ASR) systems. SITAR aggregates situational severity and cumulative extent into a unified score using a bounded recursive formulation, and is implemented as a lightweight BERT-family regressor that requires minimal runtime cost. The authors validate SITAR through extrinsic and intrinsic protocols, showing it tracks capability scaling and provides diagnostic utility across diverse datasets and models.

By Mingzhe Lu, Jinhao Liu, Yunpeng Li, Hang Su, Lichun Fan, Jian Luan
arXiv Computation and Language
Sep 3

Choosing a PEFT Variant for Per-Patient Dysarthric ASR: A Single-Speaker Case Study on Two ASR Bases

The study evaluates seven parameter‑efficient fine‑tuning (PEFT) methods—LoRA, QLoRA, AdaLoRA, DoRA, LoHA, VeRA, and VB‑LoRA—on two ASR back‑ends (Whisper‑large‑v3 and Qwen3‑ASR‑1.7B) for a single post‑stroke Hungarian male speaker with severe dysarthria. Attention‑projection adapters consistently lower character error rates (CER) on both models, with LoRA emerging as the simplest and most effective choice; QLoRA performs worse and offers no memory advantage at this scale. Full fine‑tuning yields the lowest CER, but a 115 MB LoRA that also adapts feed‑forward blocks achieves comparable accuracy with only 3.7 % of the per‑patient storage, and a 5‑minute enrollment audio captures nearly half of the zero‑shot‑to‑30‑minute CER improvement. whyItMatters":"The paper demonstrates that lightweight PEFT adapters can substantially improve dysarthric ASR performance while keeping storage and computational costs low, offering a practical path for personalized speech recognition in clinical settings."

By Bernard Muller, L\'aszl\'o T\'oth, LaVonne Roberts
arXiv AI
Jun 19

Cross-Dataset, Age, and Gender Generalization: A Comprehensive Analysis of Fine-Tuning Strategies for Low-Resource Children's ASR

arXiv:2606. 19791v1 Announce Type: cross Abstract: The challenge associated with recognizing dysarthric speech primarily arises from pronounced acoustic variability attributed to impaired articulatory precision.

By Paban Sapkota, Hemant Kumar Kathania, Mikko Kurimo, Sudarsana Reddy Kadiri, Shrikanth Narayanan
arXiv AI
Aug 25

Multi-Task Learning for Non-Canonical Phoneme Recognition via Articulatory Feature Decomposition

The paper proposes a linguistically structured multi‑task learning framework for recognizing non‑canonical phonemes by decomposing phoneme prediction into articulatory feature dimensions such as manner, place, and voicing. A hierarchical architecture with task‑specific heads and a cross‑attention fusion module is combined with semi‑supervised Momentum Pseudo‑Labeling and a cascaded training strategy that gradually introduces articulatory tasks. Experiments on the L2‑ARCTIC dataset demonstrate significant improvements over baseline models and produce interpretable error patterns aligned with phonological feature structure.

By Sophia Riaz, Haoze Zheng, Amos Roche, Miyu Zhang, Anamika Ragu, Salvatore Penachio, Kaustav Mukherjee, Aneesh Jonelagadda
arXiv Machine Learning
Sep 16

BenSParX: A Robust Explainable Machine Learning Framework for Parkinson's Disease Detection from Bengali Conversational Speech

BenSparX introduces the first Bengali conversational speech dataset for Parkinson’s disease detection and pairs it with a robust, explainable machine learning framework. The framework uses diverse acoustic features, systematic feature selection, and advanced classifiers, achieving 95.67% accuracy, 95.62% F1, and 0.990 AUC. SHAP analysis is employed to explain feature contributions, and the model outperforms state‑of‑the‑art methods on other language datasets.

By Riad Hossain, Muhammad Ashad Kabir, Arat Ibne Golam Mowla, Animesh Chandra Roy, Ranjit Kumar Ghosh