arXiv AI
Where Does the Sound Go? Tracing Acoustic Information Loss in Audio-Conditioned LLMs
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 2
Heard but Not Heeded: Paralinguistic Information Encoding and Loss in Audio-Language Models
The paper investigates whether audio‑language models capture paralinguistic cues beyond spoken content. Using the Expresso dataset and four open‑source models, the authors trace how speaking style information is encoded in the late layers of the audio encoder but is degraded before reaching the final output. They find that some models are content‑driven while others are acoustic‑driven, revealing a gap between what is encoded and what is utilized in current audio‑language models.
arXiv AI
Jul 14
Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Models
arXiv:2607. 11801v1 Announce Type: cross Abstract: Large audio-language models (LALMs) often underperform on fine-grained, non-semantic attributes of speech, such as a speaker's emotion, despite strong performance on speech content.
arXiv AI
Aug 21
Represented but Ignored: A Causal Account of Prosodic Underuse in Audio-Language Models
arXiv:2608. 19211v1 Announce Type: cross Abstract: Human speech is richly expressive, with prosody carrying linguistic and emotional information beyond the lexical content.
arXiv AI
Aug 10
Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Models
arXiv:2608. 06409v1 Announce Type: cross Abstract: Speech language models are increasingly evaluated on paralinguistic tasks by the accuracy of prompted answers, but answer accuracy combines failures at different stages of the audio-to-answer computation.
arXiv Machine Learning
3d ago
WhisTLE: Deeply Supervised, Text-Only Domain Adaptation for Small Pretrained Speech Recognition Transformers
arXiv:2509.10452v3 Announce Type: replace-cross Abstract: Pretrained automatic speech recognition (ASR) models such as Whisper perform well but still need domain adaptation to handle unseen parlance....
arXiv Machine Learning
Aug 4
Experience-Calibrated Contrastive Decoding for Mitigating Hallucinations in LM-Based Text-to-Speech
arXiv:2608. 00722v1 Announce Type: cross Abstract: Language model-based text-to-speech (LM-based TTS) remains vulnerable to speech hallucinations that deviate from the target text.