The paper examines how multilingual medical adaptation affects the internal representations of Whisper ASR models. By comparing various fine‑tuning strategies—zero‑shot decoding, English‑only, German‑only, two‑stage EN→EN+DE, and direct EN+DE fine‑tuning—it shows that fine‑tuning significantly improves performance, with the best model varying by setting. Layer‑wise encoder analysis reveals that English medical fine‑tuning drives the main representation shift, while multilingual continuation largely preserves the adapted space, and that domain and language signals remain recoverable across layers.
By Souranil Kahali, Rituparna Bose, Abner Hernandez, Tomas Arias-Vergara, Andreas Maier, Ning Ma, Paula Andrea Perez-Toro
This study evaluates automatic speech recognition (ASR) for adolescent health communication in Twi, Dagbani, and Ewe by benchmarking five ASR systems on a Bible corpus and a domain-specific ASRH dataset, then performing supervised domain adaptation with a fine‑tuned Qwen3-ASR-0.6B model. Fine‑tuning significantly lowered word and character error rates, especially for Ewe, and the adapted model was deployed in the KasaHealth voice‑first application, which received high user approval and highlighted remaining domain gaps. The work demonstrates that in‑domain data, rather than model size or computational resources, is the primary limitation for effective ASR in these languages.
By Stephen E. Moore, Akwasi Asare, Mich-Seth Owusu, Paul Azunre, Joel Budu, Lawrence A. Adu-Gyamfi
arXiv:2609.15743v1 Announce Type: new
Abstract: Automatic speech recognition (ASR) systems, trained on paired speech-text data, have been improved by leveraging language models (LMs) trained on text-...
By Hayato Futami, Tatsuya Kawahara
The paper examines how multilingual medical adaptation affects the internal representations of Whisper ASR models by performing layer‑wise encoder analysis. It compares several adaptation strategies—zero‑shot decoding, English‑only fine‑tuning, German‑only diagnostic fine‑tuning, two‑stage EN→EN+DE continuation, and direct EN+DE fine‑tuning—across different Whisper sizes, finding that fine‑tuning improves performance but the best model varies by setting. Layer‑wise results show that English medical fine‑tuning drives the main encoder shift, while multilingual continuation largely preserves the adapted representation space, with domain and language information remaining recoverable across layers.
"whyItMatters":"The study provides insight into how multilingual medical adaptation reshapes Whisper’s internal representations, guiding the selection of model sizes and fine‑tuning strategies for improved MedASR performance."
VietMed is a Vietnamese medical speech recognition dataset featuring 16 h of labeled medical speech, 1 000 h of unlabeled medical speech, and 1 200 h of unlabeled general‑domain speech. It claims to be the largest public medical ASR dataset worldwide in seven dimensions—total duration, speaker count, disease coverage, recording conditions, speaker roles, unique medical terms, and accents—and also the largest Vietnamese speech dataset by duration. The authors release pre‑trained models (w2v2‑Viet and XLSR‑53‑Viet) and fine‑tuned medical ASR models, showing that XLSR‑53‑Viet outperforms the state‑of‑the‑art XLSR‑53 by reducing WER from 51.8 % to 29.6 % on a medical test set.
whyItMatters":"The dataset and models provide the first large‑scale, publicly available Vietnamese medical ASR resource, enabling research and development of medical speech technologies in a language and domain that previously lacked sufficient data."
By Khai Le-Duc
arXiv:2609.17981v1 Announce Type: cross
Abstract: Speech Large Language Models (Speech-LLMs), typically built from a pre-trained speech encoder, a modality projector, and an LLM fine-tuned with Low-R...
By Mohan Shi, Zilai Wang, Natarajan Balaji Shankar, Kaiyuan Zhang, Eray Eren, Abeer Alwan
arXiv:2509.10452v3 Announce Type: replace-cross
Abstract: Pretrained automatic speech recognition (ASR) models such as Whisper perform well but still need domain adaptation to handle unseen parlance....
By Akshat Pandey, Karun Kumar, Raphael Tang
arXiv:2604.06487v2 Announce Type: replace
Abstract: Conventional end-to-end automatic speech recognition (ASR) systems rely on paired speech-text data for domain adaptation. Recent LLM-based ASR arch...
By Thibault Ba\~neras-Roux, Sergio Burdisso, Esa\'u Villatoro-Tello, Dairazalia S\'anchez-Cort\'es, Shiran Liu, Severin Baroudi, Shashi Kumar, Hasindri Watawana, Manjunath K E, Kadri Hacioglu, Petr Motlicek, Andreas Stolcke
arXiv:2607.01733v2 Announce Type: replace
Abstract: Speech-LLM integration has shown promising results by leveraging extensive textual pretraining, yet its specific benefits for automatic speech reco...
By Ruchao Fan, Yiming Wang, Rui Zhao, Liliang Ren, Keqi Deng, Xiaoyang Chen, Ali Zare, Bo Ren, Yuxuan Hu, Junkun Chen, Yan Huang, Yelong Shen, Jinyu Li
The paper investigates model merging as a scalable alternative to multi-task training for multi-domain automatic speech recognition (ASR). It benchmarks 11 merging algorithms across 10 European Portuguese domains, assessing in‑domain accuracy, robustness to distribution shift, and performance on English and multilingual data. The authors introduce BoostedTSV‑M, a new merging method that mitigates rank collapse through singular‑value boosting, achieving better performance than full fine‑tuning while maintaining out‑of‑distribution generalisation in a single model.
By Carlos Carvalho, Francisco Teixeira, Thomas Rolland, Alberto Abad
The paper evaluates whether audio‑language models can use multimodal clinical context to improve dysarthric speech recognition. Using a benchmark built on the Speech Accessibility Project dataset, the authors test diagnosis labels, clinician ratings, and detailed clinical descriptions as prompts for nine models. They find that these prompts yield negligible or negative effects on word error rate, though fine‑tuning with LoRA and mixed prompt formats reduces WER by 52% and benefits certain subgroups such as Down syndrome and mild‑severity speakers.
By Pehu\'en Moure, Niclas Pokel, Bilal Bounajma, Yingqiang Gao, Roman Boehringer, Longbiao Cheng, Shih-Chii Liu
arXiv:2609.14991v1 Announce Type: new
Abstract: Open Thai automatic speech recognition (ASR) is dominated by offline, Whisper-based models that read the whole utterance before transcribing, ruling ou...
By Warit Sirichotedumrong, Tanawin Samutsin, Shah Faisal Wani, Sittipong Sripaisarnmongkol, Kunat Pipatanakul