arXiv AI
3d ago

Right Answers, Inconsistent Evidence: Source-Level Counterfactual Auditing of Large Audio-Language Models

The paper introduces TRACE, a method for auditing large audio‑language models (LALMs) by deleting the queried source and each rival sound in turn to test whether views containing the target sound consistently score higher than those without it. Across 8 LALMs and 3,456 controlled mixtures, 6.3% of correct answers fail this ordering, a failure that a simple target‑only score check misses 42.5% of the time. The failure rate rises from 4.9% to 8.4% as the target signal‑to‑noise ratio drops from +5 to –5 dB, and the issue persists even under loudness and re‑pairing controls.

By Yi Zhang, Yi Li, Hongwei Du
arXiv AI
Oct 12

Does Location Matter? A Black-Box Causal Audit Reveals Model-Dependent Audio Grounding in LALM Hallucination

The paper investigates how large audio‑language models (LALMs) hallucinate objects by affirming sounds that are not present. Using a black‑box causal test, the authors isolate and remove the claimed evidence region via spectral gating and compare the effect against a random‑window control across a 2×2×2 grid on three LALMs. The study finds that localization of evidence improves detection over random removal, that raw detection values are model‑dependent, and that contrastive decoding combining blank and language‑prior contrasts can significantly boost accuracy, but a unified causal‑grounding profile does not transfer between models.

By Mohammad Khalooei, Mohammad Sabokrou
Hugging Face Trending Papers
Oct 9

Does Location Matter? A Black-Box Causal Audit Reveals Model-Dependent Audio Grounding in LALM Hallucination

The paper investigates how large audio‑language models (LALMs) hallucinate objects by affirming sounds that are not present. Using a black‑box causal test, the authors isolate evidence regions via CLAP similarity, remove them with spectral gating, and compare against random windows across a 2×2×2 grid on three LALMs. The study finds that evidence localization improves detection over random removal, that raw detection scores are model‑dependent, and that a contrastive decoding strategy combining blank and language‑prior cues boosts Qwen2‑Audio accuracy from 57.1 % to 71.9 %, outperforming Audio‑Aware Decoding alone.

Hugging Face Trending Papers
Oct 27

Beyond Answer Changes: Matched Same-Class Controls for Local Audio Evidence

The paper investigates how audio-language models react to local sound‑event edits by comparing matched same‑class and different‑class replacements. Using consistent editing parameters across 600 synthetic soundscapes and three models, the study finds that different‑class replacements cause 49.7 to 60.9 percentage points more answer changes, while same‑class replacements keep accuracy within 5.2 percentage points of the original. Soundscape‑weighted selective accuracy for the inserted class ranges from 73.7 to 81.9%.