arXiv AI September 28, 2026 By Honglei Zhang, Yuting Chen, Chenpeng Hu, Pengfei Zhou, Siyue Zhang, Yilei Shi

ReasonAudio: A Benchmark for Evaluating Reasoning Beyond Matching in Text-Audio Retrieval

ReasonAudio is a new benchmark designed to evaluate reasoning capabilities in text‑audio retrieval, addressing the gap left by existing semantic‑matching focused datasets. It tests four logical abilities—negation, temporal order, sound co‑occurrence, and sound duration—across five synthetic subtasks (1,000 queries over 10,000 composite clips) and one natural subtask (100 queries over 1,000 real‑world clips). Evaluation of 11 state‑of‑the‑art systems shows significant limitations, with the best model, OmniEmbed‑7B, scoring only 20.7 overall and 53.8% in a controlled setting, compared to 70.6% for its generative backbone and 95.6% for humans.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

benchmarks

arXiv AI

Jun 15

AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models

arXiv:2606. 14591v1 Announce Type: cross Abstract: Large Audio-Language Models (LALMs) have shown strong performance on a wide range of audio understanding tasks, yet they still struggle with complex audio reasoning.

By Hui Geng, Yi Su, Han Yin, Tianjiao Wan, Qisheng Xu, Jiaxin Chen, Zijian Gao, Hengzhu Liu, Xie Chen, Kele Xu

llms benchmarks

arXiv AI

Jun 24

Event-Grounded Question Answering over Long Audio via Structured Retrieval

arXiv:2602. 14612v4 Announce Type: replace-cross Abstract: Answering natural-language questions over multi-hour audio requires both event recognition and temporal grounding.

By Kartik Hegde, Arvind Krishna Sridhar, Naveen Vakada, Yinyi Guo, Erik Visser

llms rag nlp benchmarks data-engineering

arXiv AI

Sep 10

Audio-Maestro: Enhancing Large Audio-Language Models with Tool-Augmented Reasoning

arXiv:2510.11454v2 Announce Type: replace-cross Abstract: Recent advancements in large multimodal models (LMMs) have shown strong capabilities in audio understanding. However, most systems rely solel...

By Kuan-Yi Lee, Tsung-En Lin, Hung-Yi Lee

llms multimodal safety

arXiv Computer Vision

Sep 1

TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos

arXiv:2605.07593v2 Announce Type: replace Abstract: Real-world audio-visual understanding requires chaining evidence that is sparse, temporally dispersed, and split across the visual and auditory str...

By Hengyi Feng, Hao Liang, Mingrui Chen, Bohan Zeng, Meiyi Qiang, Zhengyang Zhao, Zimo Meng, Zeang Sheng, Wentao Zhang

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 18

Investigating Faithfulness in Large Audio Language Models

arXiv:2509. 22363v4 Announce Type: replace Abstract: Large Audio Language Models (LALMs) integrate audio encoders with pretrained Large Language Models to perform complex multimodal reasoning tasks.

By Pooneh Mousavi, Lovenya Jain, Mirco Ravanelli, Cem Subakan

llms multimodal benchmarks safety

arXiv Machine Learning

Aug 11

Assessing Factual Music Comprehension in Large Audio Language Models

arXiv:2511. 05550v3 Announce Type: replace-cross Abstract: Large audio language models (LALMs) leverage multimodal representations to generate open-ended answers to natural language queries about audio.

By Daniel Chenyu Lin, Michael Freeman, John Thickstun

llms multimodal benchmarks

One email a morning, machine-written

Related stories

AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models

Event-Grounded Question Answering over Long Audio via Structured Retrieval

Audio-Maestro: Enhancing Large Audio-Language Models with Tool-Augmented Reasoning

TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos

Investigating Faithfulness in Large Audio Language Models

Assessing Factual Music Comprehension in Large Audio Language Models