← Back to all news
arXiv Computation and Language By Jiajun Fan, Jingyuan Li, Prashanth Gurunath Shivakumar, Qi Luo, Jia-Hong Huang, M. Maruf, Roger Ren, Yile Gu, Rahul Pandey, Ge Liu, Ivan Bulyko

Can We Read the Mind of an Audio LLM? A Verbalizable, Multilingual Middle-Layer Workspace

Read the original on arXiv Computation and Language →

The study investigates the internal workings of an audio language model (Qwen3-Omni) by applying a logit lens to its middle layers. It finds that the model’s reasoning about spoken questions becomes legible in words before any token is emitted, revealing language‑agnostic, paralinguistic, and temporally distinct signals that are causally used in the network’s decision process. The authors demonstrate that these signals can be isolated and mapped to specific layers, providing a qualitative account of how the model processes audio input.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Sep 7

Tracing Audio Grounding and Answer Selection in Audio LLMs

The paper investigates how Audio Large Language Models (Audio LLMs) actually use audio input to determine answers, rather than relying on textual cues. It finds that replacing audio with silence or unrelated audio degrades performance more after training than before, that acoustic information shapes representations in early-to-middle layers and influences final predictions in middle-to-late layers, and that training impacts specific layer bands most strongly. These observations offer a mechanistic view of how training enhances the use of acoustic evidence in Audio LLMs.

By Hyebin Cho, Suho Yoo, Jihoo Jung, Joon Son Chung
llms
More like this →
arXiv Computation and Language
Sep 24

Causal Tracing of Audio-Text Fusion in Large Audio Language Models

The study applies causal tracing to large audio language models (LALMs) to uncover how they fuse acoustic and textual information. Layer‑wise analysis reveals distinct fusion strategies—progressive integration in DeSTA versus abrupt late‑stage fusion in Qwen—while token‑wise analysis identifies the final sequence token as an informational bottleneck that decisively retrieves audio content. Additionally, an attention‑like query mechanism at intermediate tokens is observed, prompting the model to pull task‑relevant audio context.

By Wei-Chih Chen, Chien-yu Huang, Hung-yi Lee
llmsmultimodal
More like this →
arXiv AI
Oct 2

Senses Wide Shut: A Representation-Action Gap in Omnimodal LLMs

The paper introduces IMAVB, a 500‑clip benchmark that tests whether omnimodal large language models can detect when a textual premise contradicts their visual or audio input. Experiments on eight open‑source models and Gemini 3.1 Pro reveal a Representation‑Action Gap: internal states encode mismatches, yet the models rarely reject false premises, exhibiting under‑rejection or over‑rejection. A probe‑guided logit adjustment improves rejection behavior, suggesting the main bottleneck is in translating perception to action rather than in perception itself.

By Trung Nguyen Quang, Yiming Gao, Fanyi Pu, Kaichen Zhang, Shuo Sun, Ziwei Liu
llmsagentsmultimodalbenchmarks
More like this →