arXiv Machine Learning

Benchmarking_Fast_Domain_Adaptation_for_Unsupervised_Speech_Units

The paper introduces ABX-Accent, a benchmark built on the AESRC dataset that evaluates how well representation learning models adapt to 10 different English accents with less than 10 hours of unlabeled data per accent. It adapts the Zero Resources Challenge ABX metrics for each accent and demonstrates a baseline using adaptive domain normalization to fine‑tune a Contrastive Predictive Coding model, achieving a 23.6% relative improvement on across‑speaker ABX scores compared to non‑adapted models. The dataset and evaluation metrics will be released publicly after the paper is accepted.

Hugging Face Trending Papers

Aug 27

Benchmarking_Fast_Domain_Adaptation_for_Unsupervised_Speech_Units

The paper introduces ABX-Accent, a benchmark for evaluating how well unsupervised speech representation learning models adapt to new accents. It uses the AESRC dataset with 10 English accents, each providing less than 10 hours of unlabeled data, and adapts the Zero Resources Challenge ABX metrics to these accents. A baseline model employing adaptive domain normalization to fine‑tune a Contrastive Predictive Coding model shows a 23.6% relative improvement on across‑speaker ABX scores compared to non‑adapted models.

benchmarks

arXiv Machine Learning

Jun 16

Contrastive Regularization for Accent-Robust ASR

arXiv:2605. 03297v2 Announce Type: replace-cross Abstract: ASR systems based on self-supervised acoustic pretraining and CTC fine-tuning achieve strong performance on native speech but remain sensitive to accent variability.

By Van-Phat Thai, Aradhya Dhruv, Duc-Thinh Pham, Sameer Alam

fine-tuning multimodal benchmarks

arXiv Machine Learning

Jul 14

An Empirical Recipe for Universal Phone Recognition

arXiv:2603. 29042v2 Announce Type: replace-cross Abstract: Phone recognition (PR) is a key enabler of multilingual and low-resource speech processing tasks, yet robust performance remains elusive.

By Shikhar Bharadwaj, Chin-Jou Li, Kwanghee Choi, Eunjung Yeo, William Chen, Shinji Watanabe, David R. Mortensen

benchmarks

arXiv Machine Learning

3d ago

WhisTLE: Deeply Supervised, Text-Only Domain Adaptation for Small Pretrained Speech Recognition Transformers

arXiv:2509.10452v3 Announce Type: replace-cross Abstract: Pretrained automatic speech recognition (ASR) models such as Whisper perform well but still need domain adaptation to handle unseen parlance....

By Akshat Pandey, Karun Kumar, Raphael Tang

llms fine-tuning multimodal

Hugging Face Trending Papers

Aug 12

Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder

Recent advances in zero-shot text-to-speech (TTS) have substantially improved speech quality and voice cloning fidelity. However, many zero-shot TTS systems still depend on audio prompt transcripts at inference time.

llms multimodal benchmarks

arXiv AI

Jun 3

Efficient ASR Training with Conversations that Never Happened

arXiv:2606. 03957v1 Announce Type: cross Abstract: Conversational ASR for lower-resource languages and niche domains is limited by the scarcity of domain-matched multi-speaker training data.

By M\'at\'e Gedeon, P\'eter Mihajlik

llms multimodal benchmarks

Hugging Face Trending Papers

Jun 2

Efficient ASR Training with Conversations that Never Happened

Conversational ASR for lower-resource languages and niche domains is limited by the scarcity of domain-matched multi-speaker training data. We propose an augmentation pipeline that generates scenario-level dialogues with participant metadata, maps speaker attributes to TTS voice profiles, and assembles synthesized utterances into speaker-aware simulated conversations.

llms multimodal benchmarks

arXiv AI

Jun 19

ZeSTA: Zero-Shot TTS Augmentation with Domain-Conditioned Training for Data-Efficient Personalized Speech Synthesis

arXiv:2603. 04219v2 Announce Type: replace-cross Abstract: We investigate the use of zero-shot text-to-speech (ZS-TTS) as a data augmentation source for low-resource personalized speech synthesis.

By Youngwon Choi, Jinwoo Oh, Hwayeon Kim, Hyeonyu Kim

rag fine-tuning multimodal

arXiv Computation and Language

3d ago

BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models

arXiv:2609.09554v1 Announce Type: new Abstract: We introduce BuzzASR, a collection of language-specialized fine-tuned Whisper models adapted for automatic speech recognition (ASR) in 102 languages. L...

By Shivam Singh, Aditya Yadavalli, Catherine Arnett, Alex Warstadt

llms nlp fine-tuning multimodal benchmarks

arXiv AI

Jul 7

TokAN: Accent Normalization Using Self-Supervised Speech Tokens

arXiv:2607. 03928v1 Announce Type: cross Abstract: Accent normalization (AN) seeks to convert non-native (L2) accented speech into standard (L1) speech while preserving speaker identity.

By Qibing Bai, Shuai Wang, Yuhan Du, Bohan Li, Yannan Wang, Haizhou Li

rag nlp reinforcement-learning fine-tuning efficiency

arXiv AI

Jun 2

PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects

arXiv:2606. 01016v1 Announce Type: cross Abstract: While End-to-End (E2E) Speech-Large Language Models (Speech-LLMs) are rapidly evolving, their evaluation methodologies remain limited to the era of simple transcription.

By Sicheng Yang, Shulan Ruan, Shiwei Wu, Yu Liu, Lu Fan, Zhi Li, You He

llms multimodal benchmarks safety

arXiv AI

Aug 6

Breaking the Curse ofMultilinguality inMany-to-Many Speech-to-Text Translation via a Resource-AwareMixture of Speech Encoders

arXiv:2608. 04586v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have achieved significant success in speech-to-text translation (S2TT).

By Yexing Du, Kaiyuan Liu, Youcheng Pan, Bo Yang, Chengpeng Fu, Yu Wang, Ming Liu

llms multimodal benchmarks safety

Related stories