← Back to all news
arXiv AI By Bhuvan Koduru, Dareen Safar B Alharthi, Rita Singh, Bhiksha Raj

Heard but Not Heeded: Paralinguistic Information Encoding and Loss in Audio-Language Models

Read the original on arXiv AI →

The paper investigates whether audio‑language models capture paralinguistic cues beyond spoken content. Using the Expresso dataset and four open‑source models, the authors trace how speaking style information is encoded in the late layers of the audio encoder but is degraded before reaching the final output. They find that some models are content‑driven while others are acoustic‑driven, revealing a gap between what is encoded and what is utilized in current audio‑language models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 12

AudioTETRIS: Token Elimination via Transitional Importance Signals for Efficient Audio LLMs

AudioTETRIS is a training‑free, two‑stage token elimination framework designed for large audio language models. It leverages the geometry of audio encoder representations as an acoustic prior to compute per‑token importance scores, which guide progressive top‑k token selection across decoder layers. Experiments on Qwen2‑Audio‑7B, Phi‑4‑Multimodal, and Audio Flamingo 3 show that AudioTETRIS retains near‑full performance while using only 25% of audio tokens and outperforms existing training‑free baselines.

By Shreya Sajid, Anna Vettoruzzo, Joaquin Vanschoren
llmsmultimodal
More like this →
arXiv Computation and Language
Sep 4

Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe

The paper introduces an Encoding Probe that reconstructs language model representations using interpretable features, addressing limitations of traditional decoding probes such as incomparable feature contributions and correlation effects. It evaluates this approach on text and speech transformer models, examining features from acoustics, phonetics, syntax, lexicon, and speaker identity. Findings reveal that speaker-related effects vary with training objectives and datasets, while syntactic and lexical features independently contribute to reconstruction, offering a complementary perspective on model interpretation.

By Gaofei Shen, Martijn Bentum, Tomas O. Lentz, Afra Alishahi, Grzegorz Chrupa{\l}a
llms
More like this →