arXiv AI
3d ago

Right Answers, Inconsistent Evidence: Source-Level Counterfactual Auditing of Large Audio-Language Models

The paper introduces TRACE, a method for auditing large audio‑language models (LALMs) by deleting the queried source and each rival sound in turn to test whether views containing the target sound consistently score higher than those without it. Across 8 LALMs and 3,456 controlled mixtures, 6.3% of correct answers fail this ordering, a failure that a simple target‑only score check misses 42.5% of the time. The failure rate rises from 4.9% to 8.4% as the target signal‑to‑noise ratio drops from +5 to –5 dB, and the issue persists even under loudness and re‑pairing controls.

By Yi Zhang, Yi Li, Hongwei Du