arXiv:2603. 09714v2 Announce Type: replace-cross Abstract: While multi-audio understanding is critical for large audio-language models (LALMs), it remains underexplored.
By Chih-Kai Yang, Yun-Shao Tsai, Yu-Kai Guo, Ping-Le Tsai, Yen-Ting Piao, Hung-Wei Chen, Ting-Lin Hsiao, Yun-Man Hsu, Ke-Han Lu, Hung-yi Lee
arXiv:2609.22851v2 Announce Type: cross
Abstract: Large Audio Language Models (LALMs) utilize either continuous features or discrete tokens, yet the optimal representation paradigm for general audio...
By Jing Peng, Zichao Nie, Zhisheng Zhang, Jingran Xie, Zhiyong Wu
arXiv:2607. 05196v1 Announce Type: cross Abstract: Audio intelligence involves understanding, reasoning about, and generating both audio and speech.
By Zhifeng Kong, Sang-gil Lee, Jaehyeon Kim, Boxin Wang, Zihan Liu, Sungwon Kim, Yang Chen, Arushi Goel, Rajarshi Roy, Wenliang Dai, Zhuolin Yang, Yangyi Chen, Dongfu Jiang, Sreyan Ghosh, Tuomas Rintamaki, Andrew Tao, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
arXiv:2510.11454v2 Announce Type: replace-cross
Abstract: Recent advancements in large multimodal models (LMMs) have shown strong capabilities in audio understanding. However, most systems rely solel...
By Kuan-Yi Lee, Tsung-En Lin, Hung-Yi Lee
Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM.
The paper introduces CUES, a lightweight heuristic for selecting encoder combinations in large audio‑language models by estimating complementarity through Pearson correlations of single‑encoder performance profiles. Using a frozen SmolLM2‑135M backbone, CUES consistently identifies optimal encoder sets for each track on the XARES‑LLM benchmark without requiring fusion training or test data. On broad audio tasks, CUES selects a diverse trio of encoders, improving performance by 4.3% over Whisper‑medium, while on text generation it opts for a focused speech‑only pair, outperforming mHuBERT‑147 by 6.3%. The results illustrate how correlation signals guide a diversity–interference trade‑off across different task families.
By Pei-Jun Liao, Hung-Shin Lee, Wenze Ren, Kuo-Hsuan Hung, Hung-yi Lee, Hsin-Min Wang
The paper introduces a compression framework for the Whisper automatic speech recognition model that jointly optimizes six deployment dimensions—model size, temporal resolution, encoder token stride, low‑rank adaptation capacity, weight precision, and sparsity pattern—using NSGA‑III. The optimization targets three objectives: word error rate, inference FLOPs, and memory footprint. Evaluating 1,680 configurations, the study identifies compression combinations that outperform single‑axis scaling and notes that 1:4 structured sparsity cannot maintain acceptable accuracy within the tested budgets.
By Vyom Agarwal, Mokshda Gangrade, Siddharth Pal, Jerry Wu
Multi-encoder fusion extends Large Audio-Language Models (LALMs) beyond speech-centric recognition, but selecting encoders via intuition or exhaustive search often introduces redundant representations...
AU-Harness is an open-source toolkit designed to streamline and standardize the evaluation of Audio Language Models (LALMs). It addresses three key limitations of existing frameworks: slow processing pipelines, lack of multi-turn dialogue support, and absence of a scalable, unified evaluation system. The toolkit achieves a 77% reduction in evaluation time, enabling large-scale studies and providing standardized protocols for fair model comparison and in-depth analysis of audio reasoning capabilities.
By Hoang Nguyen, Sidharth Surapaneni, Akshay Kalkunte, Jash Mehta, Aman Tiwari, Oluwanifemi Bamgbose, Khyati Mahajan, Jash Shah, Shruthan Radhakrishna, Sathwik Tejaswi Madhusudhan, Vikas Yadav, Sai Rajeswar
Mizar is a 159.3‑million‑parameter audio‑language model designed for devices with limited memory and computation. It couples a compact CED‑Small audio encoder with SmolLM2‑135M via a frequency‑merging mapper and is trained in three stages—audio‑language alignment, audio‑dependent fine‑tuning, and post‑training—to improve performance on audio‑question tasks. Across five random seeds, Mizar outperforms all other sub‑200M‑parameter ALMs on MMAU, MMAR, and ADQA‑clean, achieving mean accuracies of 52.92%, 42.42%, and 36.02% respectively, while enabling local inference on a single CPU with an average latency of 1.09 seconds for MMAU questions.
By Kaiyang Li, Shaobo Han, Yue Tian, Shihao Ji
arXiv:2609.14467v1 Announce Type: cross
Abstract: Automating clinical documentation from long-form doctor-patient conversations remains challenging for modern audio-language models. While cascaded AS...
By Ziyu Zhang, Mingchen Shao, Wenjie Tian, Tianlun Zuo, Longhao Li, Lei Xie
The paper proposes a hierarchical conditioning approach for joint Acoustic Scene Classification and Audio Tagging, predicting scene categories directly from intermediate event probabilities. Extensive experiments on three backbones and varied loss‑weighting settings show that this method matches or surpasses traditional multi‑head models while enabling synchronized dual‑task learning for certain beta values. An interpretability framework combining information bottleneck analysis, manifold geometry, and Formal Concept Analysis on TF‑IDF event representations demonstrates that hierarchical conditioning reduces conditional entropy, increases mutual information, and improves latent cluster separability, yielding a more structured and semantically faithful feature space.
By Claudia Montero-Ram\'irez, Alba Mart\'inez-Serrano, Carmen Pel\'aez-Moreno