arXiv Computation and Language By Michael Picheny

Whisper-Based Speech Transcription from Videos Across Multiple Languages for Cross-Cultural Understanding

Read the original on arXiv Computation and Language →

The paper introduces methods to improve speech recognition for multilingual video transcription using Whisper-based tools, targeting cross‑cultural understanding. It reports an average transcription error rate of 30% across seven languages, which can be lowered to 20% with modest fine‑tuning. The authors also release associated speech and metadata to aid community refinement of these techniques.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language
Aug 27

Fine-Tuning Whisper for Automatic Speech Recognition in Baniwa: A Preliminary Study

The study fine‑tunes the Whisper Small model for Automatic Speech Recognition (ASR) in Baniwa, an indigenous Arawakan language. Using a 0.54‑hour corpus of 1,373 manually transcribed recordings, the fine‑tuned model achieved a Word Error Rate of 37.5% and a Character Error Rate of 7.45%. These results provide an initial baseline for Baniwa ASR and suggest that multilingual foundation models can be adapted to extremely low‑resource languages.

By Leonardo Duart, Tiago Fonseca, Thiago Chac\'on
Hugging Face Trending Papers
Jun 2

Efficient ASR Training with Conversations that Never Happened

Conversational ASR for lower-resource languages and niche domains is limited by the scarcity of domain-matched multi-speaker training data. We propose an augmentation pipeline that generates scenario-level dialogues with participant metadata, maps speaker attributes to TTS voice profiles, and assembles synthesized utterances into speaker-aware simulated conversations.

arXiv Computation and Language
Sep 10

Qwen-Audio-3.0-ASR Technical Report

The Qwen-Audio-3.0-ASR Technical Report introduces a Mixture-of-Experts large language model-based automatic speech recognition system that addresses real‑world production challenges such as regional dialects, dynamic entities, hotwords, long‑range context, and disfluent speech. Built on the Qwen backbone and trained on tens of millions of hours of speech data, it supports transcription in 30 languages and 16 Chinese dialects, and offers industry‑domain entity recognition, hierarchical hotword customization, single‑pass polishing, and long‑audio contextual modeling. A streaming variant, Qwen-Audio-3.0-ASR-Streaming, is also presented for low‑latency applications, with evaluations showing state‑of‑the‑art performance against leading commercial systems.

By Chuanmeng Bian, Daren Chen, Peixin Chen, Zhigao Chen, Zhiyun Fan, Zhifu Gao, Bo Gong, Qing Gu, Jiajun He, Yawei Hu, Yunjie Ji, Jingbei Li, Xiangang Li, Xu Li, Zengxi Li, Zheng Li, Chengdong Liang, Baiji Liu, Ying Liu, Bin Ma, Yiping Peng, Yuezhang Peng, Zhendong Peng, Yu Pu, Yang Shi, Xin Shu, Jian Tang, Biao Tian, Peiyao Wang, Tianzi Wang, Wen Wang, Wupeng Wang, Cheng Wen, Yuzhong Wu, Zijian Xia, Yunchong Xiao, Nan Yang, Jianwei Yu, Jixing Yu, Binbin Zhang, Lei Zhang, Sitong Zhao, Guangdong Zhou, Yuan Zhou, Jianheng Zhuo