arXiv:2609.17981v1 Announce Type: cross Abstract: Speech Large Language Models (Speech-LLMs), typically built from a pre-trained speech encoder, a modality projector, and an LLM fine-tuned with Low-R...
arXiv:2604.06487v2 Announce Type: replace Abstract: Conventional end-to-end automatic speech recognition (ASR) systems rely on paired speech-text data for domain adaptation. Recent LLM-based ASR arch...
arXiv:2609.15743v1 Announce Type: new Abstract: Automatic speech recognition (ASR) systems, trained on paired speech-text data, have been improved by leveraging language models (LMs) trained on text-...
arXiv:2509.10452v3 Announce Type: replace-cross Abstract: Pretrained automatic speech recognition (ASR) models such as Whisper perform well but still need domain adaptation to handle unseen parlance....
The paper introduces ABX-Accent, a benchmark for evaluating how well unsupervised speech representation learning models adapt to new accents. It uses the AESRC dataset with 10 English accents, each providing less than 10 hours of unlabeled data, and adapts the Zero Resources Challenge ABX metrics to these accents. A baseline model employing adaptive domain normalization to fine‑tune a Contrastive Predictive Coding model shows a 23.6% relative improvement on across‑speaker ABX scores compared to non‑adapted models.
The paper introduces ABX-Accent, a benchmark built on the AESRC dataset that evaluates how well representation learning models adapt to 10 different English accents with less than 10 hours of unlabeled data per accent. It adapts the Zero Resources Challenge ABX metrics for each accent and demonstrates a baseline using adaptive domain normalization to fine‑tune a Contrastive Predictive Coding model, achieving a 23.6% relative improvement on across‑speaker ABX scores compared to non‑adapted models. The dataset and evaluation metrics will be released publicly after the paper is accepted.
arXiv:2601. 18904v3 Announce Type: replace-cross Abstract: Generative AI for speech and audio is increasingly expected to serve users across languages, cultures, and communities, yet current auditory Large Language Models (LLMs) are still largely trained and evaluated on high-resource data.
Connecting a pre-trained speech encoder to a Large Language Model (LLM) is the standard architecture for building Speech LLMs. However, a structural misalignment exists between the encoder and the LLM.
arXiv:2609.23825v1 Announce Type: new Abstract: We present a comprehensive benchmark of Federated Learning (FL) for multilingual Automatic Speech Recognition (ASR), evaluating four Speech-LLM archite...
The paper introduces asymmetric classifier‑free guidance (CFG) for target‑speaker ASR using Whisper, where a speaker‑conditioned branch predicts the target transcript and a speaker‑unconditioned branch predicts serialized multi‑speaker transcripts. CFG modulates the influence of speaker conditioning during decoding via a single guidance scale, which is first set globally on development data and then refined per utterance by a lightweight encoder‑based predictor while keeping the recognition model fixed. The resulting system yields up to 21.8% relative WER reduction over a condition‑only baseline and 5.6% over standard conditional decoding under domain shifts.
The paper introduces Hybrid Search, a method that refines warm-initialized large language model (LLM) based automatic speech recognition (ASR) systems by exploiting interactions between ASR hidden states and the base LLM’s hidden states. By identifying tokens with high semantic dependence and selectively correcting them, the approach surpasses traditional global LLM‑correction techniques such as rescoring and late fusion. The study demonstrates that even after warm initialization, LLM‑based ASR models can further benefit from their base LLM during inference.
arXiv:2606. 11033v1 Announce Type: cross Abstract: Recent efforts to extend large language models (LLMs) to speech inputs typically rely on cascaded ASR-LLM pipelines, end-to-end speech-language models, or bridge/distillation-based adaptation.