arXiv Computation and Language September 28, 2026 By Yiwen Guan, Jacob Whitehill

Asymmetric Classifier-Free Guidance for Target-Speaker ASR

Read the original on arXiv Computation and Language →

The paper introduces asymmetric classifier‑free guidance (CFG) for target‑speaker ASR using Whisper, where a speaker‑conditioned branch predicts the target transcript and a speaker‑unconditioned branch predicts serialized multi‑speaker transcripts. CFG modulates the influence of speaker conditioning during decoding via a single guidance scale, which is first set globally on development data and then refined per utterance by a lightweight encoder‑based predictor while keeping the recognition model fixed. The resulting system yields up to 21.8% relative WER reduction over a condition‑only baseline and 5.6% over standard conditional decoding under domain shifts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

multimodal

arXiv Computation and Language

2d ago

Inference-Time Target Speaker Unlearning in LLM-Based Automatic Speech Recognition

The paper introduces a new target‑speaker unlearning task for automatic speech recognition (TSU‑ASR) that allows certain speakers to opt out of transcription while still indicating their presence. A lightweight Enrollment‑Conditioned Gating (ECG) module is added to a frozen dual‑stream speech LLM, enabling dynamic unlearning of new opt‑out speakers during inference. Experiments on AMI and AliMeeting datasets show significant drops in transcription accuracy for opt‑out speakers while preserving performance for retained speakers.

By Bo Su, Yueru Yan, Thai Le

llms multimodal safety

arXiv AI

Aug 21

Towards Quantifying Benchmark Optimization in ASR Models

arXiv:2608. 19936v1 Announce Type: cross Abstract: Public benchmarks are important measures of Automatic Speech Recognition (ASR) model capabilities.

By Theo Lebryk, David Ayllon, Alice Baird, Jakub Piotr C{\l}apa, Jens Madsen, Panagiotis Tzirakis

multimodal benchmarks

arXiv Computation and Language

Sep 17

A Probe Shift Is Not a Fairness Fix: The Limits of Representation Steering in Speech Models

arXiv:2609.18533v1 Announce Type: new Abstract: Automatic speech recognition (ASR) systems exhibit unequal error rates across speaker groups, motivating interventions on their internal representation...

By Nicolas Bourrel, Abderrahmane Issam, Gerasimos Spanakis

multimodal safety

arXiv Machine Learning

Sep 10

WhisTLE: Deeply Supervised, Text-Only Domain Adaptation for Small Pretrained Speech Recognition Transformers

arXiv:2509.10452v3 Announce Type: replace-cross Abstract: Pretrained automatic speech recognition (ASR) models such as Whisper perform well but still need domain adaptation to handle unseen parlance....

By Akshat Pandey, Karun Kumar, Raphael Tang

llms fine-tuning multimodal

arXiv Computation and Language

Sep 17

Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs

arXiv:2609.17981v1 Announce Type: cross Abstract: Speech Large Language Models (Speech-LLMs), typically built from a pre-trained speech encoder, a modality projector, and an LLM fine-tuned with Low-R...

By Mohan Shi, Zilai Wang, Natarajan Balaji Shankar, Kaiyuan Zhang, Eray Eren, Abeer Alwan

llms fine-tuning multimodal benchmarks

arXiv AI

Jul 10

PS4: Proxy-Supervised Joint Training for Real Target Speaker Extraction

arXiv:2607. 08111v1 Announce Type: cross Abstract: Training target speaker extraction (TSE) models for real conversational mixtures remains challenging because large-scale training corpora and clean target speech for supervision are unavailable.

By Wanyi Ning, Wei Zhou, Yingpeng Li, Yinshang Guo, Haitao Qian, Yiming Cheng

fine-tuning multimodal benchmarks

One email a morning, machine-written

Related stories

Inference-Time Target Speaker Unlearning in LLM-Based Automatic Speech Recognition

Towards Quantifying Benchmark Optimization in ASR Models

A Probe Shift Is Not a Fairness Fix: The Limits of Representation Steering in Speech Models

WhisTLE: Deeply Supervised, Text-Only Domain Adaptation for Small Pretrained Speech Recognition Transformers

Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs

PS4: Proxy-Supervised Joint Training for Real Target Speaker Extraction