arXiv AI September 4, 2026 By Chan-Jan Hsu, Jaeyeon Kim, Chao-Han Huck Yang, Shinji Watanabe, Hung-yi Lee, Carlos Busso

Listen to the Latents: Self-Correcting Speech Recognition in Large Audio Language Models Through Hidden-State Interactions

The paper introduces Hybrid Search, a method that refines warm-initialized large language model (LLM) based automatic speech recognition (ASR) systems by exploiting interactions between ASR hidden states and the base LLM’s hidden states. By identifying tokens with high semantic dependence and selectively correcting them, the approach surpasses traditional global LLM‑correction techniques such as rescoring and late fusion. The study demonstrates that even after warm initialization, LLM‑based ASR models can further benefit from their base LLM during inference.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 16

LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization

arXiv:2506. 16738v2 Announce Type: replace-cross Abstract: With the rapid progress of speech language models (SLMs), discrete speech tokens have emerged as a core interface between speech and text, enabling unified modeling across modalities.

By Daejin Jo, Jeeyoung Yun, Byungseok Roh, Sungwoong Kim

llms nlp efficiency multimodal safety

arXiv AI

Aug 21

Towards Quantifying Benchmark Optimization in ASR Models

arXiv:2608. 19936v1 Announce Type: cross Abstract: Public benchmarks are important measures of Automatic Speech Recognition (ASR) model capabilities.

By Theo Lebryk, David Ayllon, Alice Baird, Jakub Piotr C{\l}apa, Jens Madsen, Panagiotis Tzirakis

multimodal benchmarks

arXiv Computation and Language

6d ago

A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification

arXiv:2512.07571v3 Announce Type: replace Abstract: This paper presents a simple method that allows to easily enhance textual pre-trained large language models with speech information, when fine-tune...

By Nicolas Calbucura, Jose Guillen, Valentin Barriere

llms rag nlp fine-tuning multimodal

arXiv AI

Aug 11

VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference

arXiv:2608. 08569v1 Announce Type: new Abstract: Recent advancements in Speech Large Language Models have demonstrated remarkable capabilities in understanding complex audio tasks.

By Wenxu Jia, Dongjie Fu, Xize Cheng, Fangming Feng, Linjun Li, Wenshi Chen, Yingming Li, Zhou Zhao, Tao Jin

llms efficiency multimodal benchmarks safety

arXiv AI

Aug 10

MetaSICL: Globalizing Auditory LLMs for Underserved Speakers and Languages via Meta Speech In-Context Learning

arXiv:2601. 18904v3 Announce Type: replace-cross Abstract: Generative AI for speech and audio is increasingly expected to serve users across languages, cultures, and communities, yet current auditory Large Language Models (LLMs) are still largely trained and evaluated on high-resource data.

By Haolong Zheng, Siyin Wang, Zengrui Jin, Mark Hasegawa-Johnson

llms reinforcement-learning fine-tuning multimodal

arXiv AI

Aug 21

Towards Audio Token Compression in Large Audio Language Models

arXiv:2511. 20973v2 Announce Type: replace-cross Abstract: Large Audio Language Models (LALMs) deliver strong performance across speech and audio tasks, but their audio encoders generate high-rate token sequences (e.

By Saurabhchand Bhati, Samuel Thomas, Hilde Kuehne, Rogerio Feris, James Glass

llms computer-vision fine-tuning multimodal

One email a morning, machine-written

Related stories

LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization

Towards Quantifying Benchmark Optimization in ASR Models

A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification

VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference

MetaSICL: Globalizing Auditory LLMs for Underserved Speakers and Languages via Meta Speech In-Context Learning

Towards Audio Token Compression in Large Audio Language Models