arXiv AI August 12, 2026 By Gaopeng Xu, Zhenyu Wang, Zheng Xue, Yinfeng Xia, Haitao Yao

Whisper-Aware LLM: Self-Supervised Uncertainty Learning for Robust Whispered Speech Recognition

arXiv:2608. 10836v1 Announce Type: cross Abstract: The signal ambiguity of whispered speech drives ASR systems toward two opposing failure modes: failing to capture whispered speech or hallucinatory transcription of noise.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 8

Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders

arXiv:2606. 07473v1 Announce Type: cross Abstract: Whisper, a widely adopted ASR model, is known to suffer from hallucinations - coherent transcriptions generated for non-speech audio entirely disconnected from the input.

By Georgii Aparin, Vadim Popov, Tasnima Sadekova, Assel Yermekova

fine-tuning multimodal safety

arXiv AI

Sep 7

Reducing Hallucinated Transcripts in Whisper via Hallucination Space Projection

The paper introduces a training‑free, inference‑time technique to curb hallucinated transcripts in Whisper by projecting decoder activations away from a low‑rank hallucination subspace derived from non‑speech data. Two variants are tested: an always‑on projection that dramatically lowers hallucination rates on non‑speech benchmarks, and a gated version that applies the projection only when non‑speech is predicted, achieving a smaller but still significant reduction. On LibriSpeech, the gated method slightly increases word error rate but keeps false‑rejection rates low, demonstrating a controllable trade‑off between hallucination suppression and recognition accuracy.

By Maryam Abbasihafshejani, Murtuza Jadliwala

multimodal benchmarks safety

arXiv Computation and Language

Sep 15

SURE-Voice: A Front-End Baseline for Speech-Evidence Filtering in Speech LLMs

arXiv:2608.27783v3 Announce Type: replace-cross Abstract: Speech language models (speech LLMs) can generate plausible outputs from audio that contains no usable speech evidence. We study this failure...

By Mengzhe Geng

llms multimodal benchmarks

arXiv Machine Learning

Jun 16

Semi-Supervised Speech Confidence Detection using Pseudo-Labelling and Whisper Embeddings

arXiv:2606. 16505v1 Announce Type: cross Abstract: Understanding speaker confidence is crucial in educational settings, as it can enhance personalised feedback and improve learning outcomes.

By Adam Wynn, Jingyun Wang, Xiangyu Tan

rag multimodal

arXiv AI

Sep 10

Where Does the Sound Go? Tracing Acoustic Information Loss in Audio-Conditioned LLMs

arXiv:2609.05871v1 Announce Type: cross Abstract: Audio-conditioned language models often underuse acoustic cues such as prosody, emotion, and non-speech sounds, raising the question of whether ASR-s...

By Song-ha Jo, Sehyun Lee, Soyoon Kim, Jaesik Choi, Sanghyuk Choi

llms multimodal safety

arXiv AI

Aug 5

GAP-URGENet: A Generative-Predictive Fusion Framework for Universal Speech Enhancement

arXiv:2604. 01832v1 Announce Type: cross Abstract: We introduce GAP-URGENet, a generative-predictive fusion framework developed for Track 1 of the ICASSP 2026 URGENT Challenge.

By Xiaobin Rong, Yushi Wang, Zheng Wang, Jing Lu

One email a morning, machine-written

Related stories

Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders

Reducing Hallucinated Transcripts in Whisper via Hallucination Space Projection

SURE-Voice: A Front-End Baseline for Speech-Evidence Filtering in Speech LLMs

Semi-Supervised Speech Confidence Detection using Pseudo-Labelling and Whisper Embeddings

Where Does the Sound Go? Tracing Acoustic Information Loss in Audio-Conditioned LLMs

GAP-URGENet: A Generative-Predictive Fusion Framework for Universal Speech Enhancement