arXiv AI
Where Does the Sound Go? Tracing Acoustic Information Loss in Audio-Conditioned LLMs
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 2
Heard but Not Heeded: Paralinguistic Information Encoding and Loss in Audio-Language Models
The paper investigates whether audio‑language models capture paralinguistic cues beyond spoken content. Using the Expresso dataset and four open‑source models, the authors trace how speaking style information is encoded in the late layers of the audio encoder but is degraded before reaching the final output. They find that some models are content‑driven while others are acoustic‑driven, revealing a gap between what is encoded and what is utilized in current audio‑language models.
arXiv Computation and Language
Oct 19
VocalGrad: Evaluating Acoustic Perception in Audio Language Models
arXiv:2610.23745v1 Announce Type: cross Abstract: Recent large audio language models (LALMs) have extended their capabilities beyond lexical speech understanding to tasks requiring paralinguistic and...
arXiv AI
Jul 14
Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Models
arXiv:2607. 11801v1 Announce Type: cross Abstract: Large audio-language models (LALMs) often underperform on fine-grained, non-semantic attributes of speech, such as a speaker's emotion, despite strong performance on speech content.
arXiv AI
Aug 21
Represented but Ignored: A Causal Account of Prosodic Underuse in Audio-Language Models
arXiv:2608. 19211v1 Announce Type: cross Abstract: Human speech is richly expressive, with prosody carrying linguistic and emotional information beyond the lexical content.
arXiv AI
Sep 28
Audio LLMs Know When They Can't Hear You
The paper investigates whether audio large language models (Audio LLMs) can detect when their own transcriptions are unreliable. It finds that the models are poor at self-assessment and that existing methods offer limited detection. By leveraging audio-encoder representations, the authors develop a lightweight predictor that accurately flags unreliable transcriptions and can prompt user clarification without altering the underlying model.
arXiv AI
Aug 10
Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Models
arXiv:2608. 06409v1 Announce Type: cross Abstract: Speech language models are increasingly evaluated on paralinguistic tasks by the accuracy of prompted answers, but answer accuracy combines failures at different stages of the audio-to-answer computation.