← Back to all news
arXiv Computation and Language

Heard But Not Said: Prompting Understates Temporal Information of Musical Moments in Frozen Large Audio-Language Models

Large audio‑language models (LALMs) can understand music but struggle to pinpoint a described moment in time. Using a paired‑query probe on Slakh2100 stem activations, researchers attached state and change queries to the same audio span and varied preceding context. Across frozen LALMs, different extraction methods (full‑track prompting, windowed yes/no scoring, and attention) revealed that state queries perform well without context, while change queries benefit from preceding audio in most short‑segment models; however, 30‑second padded encoders show no comparable recovery, and prompting or attention alone expose almost none of this capability.

Related stories

arXiv AI
Sep 2

MusTBench: Benchmarking and Advancing Temporal Grounding in Music LLMs

MusTBench is a music‑expert‑validated benchmark that evaluates temporal grounding in Large Audio‑Language Models (LALMs) through five temporally grounded question‑answering tasks. The paper also introduces MusT, a four‑stage optimization recipe—music encoder adaptation, LLM adaptation, supervised fine‑tuning, and RL‑based optimization—to improve temporal grounding. Experiments show that current LALMs struggle with precise temporal grounding, while MusT yields significant improvements, highlighting temporal grounding as a key missing capability in these models.

By Daeyong Kwon, Qiyu Wu, Shinobu Kuriya, Junghyun Koo, Shuyang Cui, Zhi Zhong, Wei-Hsiang Liao, Hiromi Wakaki, Yuki Mitsufuji
llmsreinforcement-learningfine-tuningbenchmarks
More like this →
arXiv Machine Learning
Jun 17

A Closer Look at Failure Modes in Temporal Understanding of Large Audio-Language Models

arXiv:2606. 17417v1 Announce Type: cross Abstract: Large Audio Language Models (LALMs) achieve strong performance on a variety of audio understanding tasks but continue to struggle with temporal reasoning, a fundamental capability central to human auditory perception.

By Apoorva Kulkarni, Kaousheik Jayakumar, Sreyan Ghosh, Sarah Wiegreffe, Dinesh Manocha, Ramani Duraiswami
llmsfine-tuningbenchmarks
More like this →
arXiv Machine Learning
Sep 2

MRMAD: A Multi-Round Multi-Audio Benchmark for Evaluating Acoustic Degradation Perception in Large Audio-Language Models

arXiv:2608.22236v2 Announce Type: replace-cross Abstract: Large audio-language models (LALMs) have shown promising progress in understanding speech, music, and general sound events, yet their ability...

By Yize Li, Ningyuan Yang, Sile Yin, Sindhuja Thogarrati, Sung-En Chang, Andrew C. Singer, Xue Lin, Chuan-Che Huang, Shuo Zhang
llmsbenchmarks
More like this →
arXiv Machine Learning
Sep 14

What Did the MLLM Hear? Token-Level Spectro-Temporal Grounding for Audio MLLM Explainability

The paper introduces STAG, a post‑hoc framework that provides token‑level spectro‑temporal grounding for captions produced by audio‑based multimodal large language models (MLLMs). STAG estimates temporal support for each token via vocabulary projections of encoded audio, measures frequency‑band relevance through controlled spectral occlusion, and fuses these signals into a spectro‑temporal relevance map. Evaluations across ten explanation methods and four grounding benchmarks show that STAG achieves superior event‑localization performance on every dataset, and counterfactual deletion experiments confirm that removing the identified evidence selectively reduces model confidence and often eliminates the corresponding event from regenerated captions.

By Lucia Cascone, Valeria Fraenza, Michele Nappi, Fabio Narducci, Benedetto Simone
llmsmultimodalbenchmarks
More like this →