One Waveform, Two Stories: Benchmarking Speech-Acoustic Conflict in Audio-Language Models

The paper introduces SAC‑Bench, a benchmark of 616,031 audio clips that test audio‑language models (ALMs) on how they handle conflicts between spoken claims and acoustic evidence. It covers 188 sound‑event classes across seven semantic, temporal, and spatial tasks, pairing environmental recordings with either compatible or contradictory synthesized spoken claims and including audio‑only controls. Evaluation of five open‑weight ALMs shows that high audio‑only accuracy often coexists with significant drops in performance under conflicting claims, revealing gaps in conflict robustness and the mixed effects of prompt interventions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms
• benchmarks