arXiv Computation and Language

Mizar: A 159M-Parameter Audio-Language Model for Audio Understanding

Mizar is a 159.3‑million‑parameter audio‑language model designed for devices with limited memory and computation. It couples a compact CED‑Small audio encoder with SmolLM2‑135M via a frequency‑merging mapper and is trained in three stages—audio‑language alignment, audio‑dependent fine‑tuning, and post‑training—to improve performance on audio‑question tasks. Across five random seeds, Mizar outperforms all other sub‑200M‑parameter ALMs on MMAU, MMAR, and ADQA‑clean, achieving mean accuracies of 52.92%, 42.42%, and 36.02% respectively, while enabling local inference on a single CPU with an average latency of 1.09 seconds for MMAU questions.

arXiv Computation and Language
Sep 28

Symbiotic Architecture for Post-Hoc Audio Extension of Frozen Language Models

The paper introduces a symbiotic architecture that equips large language models with audio‑understanding abilities without fine‑tuning their weights. It uses an injector module to write audio‑conditioned vectors into the LLM’s key‑value cache, allowing the model to act as an audio language model while keeping the backbone unchanged. The approach improves scalability—since injection cost depends on the injector width—and preserves the LLM’s original text performance, outperforming conventional frozen‑LLM methods and approaching fine‑tuned ALM results on audio tasks.

By Yotaro Kubo, Qi Sun, Yujin Tang
arXiv AI
Oct 1

UniAE-MoE: A Unified Audio Encoder via Mixture of Experts

UniAE-MoE is a unified audio encoder that uses a Mixture-of-Experts architecture to model cross‑domain audio representations. It integrates encoder components from Qwen2‑Audio and Audio‑Flamingo 3, enhances them with SwiGLU and shared experts, and applies a two‑stage instruction‑tuning strategy along with task‑specific data scaling. The model achieves state‑of‑the‑art results on the XARES‑LLM benchmark (0.802) and tops the Interspeech 2026 Audio Encoder Capability Challenge, demonstrating strong generalization across speech, music, and general audio tasks.

By Shengbo Cai, Zhisheng Zhang, Zichao Nie, Jing Peng, Jingran Xie, Zhiyong Wu
arXiv Computation and Language
Oct 23

TORUS: A Unified Closed-Loop Benchmark for Unified Audio Models

The paper introduces TORUS, a closed‑loop benchmark designed to evaluate unified audio models on their ability to understand the audio they generate or edit. TORUS contains 1,053 questions across 21 tasks and 8 audio domain settings, covering all three audio types. The authors evaluate both unified models and cascades of specialized models, finding that the best unified models answer only about 52.4% of questions about their own outputs, while GPT‑Audio achieves 53.6%.

By Aryan Vijay Bhosale, Harshit Rajgarhia, Abhishek Mukherji, Dinesh Manocha
arXiv Computation and Language
6d ago

Machine Hearing: Can Audio LLMs Generalize Beyond Human Auditory Perception?

The paper introduces the Machine Hearing Benchmark (MHB), a dataset of 6,128 instances covering 13 tasks across seven domains with diverse audio regimes, from extremely low to high sampling rates and varying sequence lengths. Evaluations show that existing Audio LLMs have limited zero‑shot transfer to these machine‑hearing tasks, and supervised adaptation alone does not close the performance gap, indicating a mismatch between current audio front ends and machine‑hearing inputs. To bridge this gap, the authors propose MachEar, a lightweight encoder extension that enhances a pretrained Audio LLM’s machine‑hearing capabilities while largely preserving its original audio understanding performance.

By Liwei Liu, Jun Tao, Xingyuan Li, Xuenan Xu, Chen Zhang, Junyi Wang, Boya Zhang, Xiaoyu Yang, Zhen-Hua Ling, Wen Wu, Chao Zhang
arXiv Machine Learning
Oct 7

AdaLoop: Adaptive-Depth Latent Reasoning for Audio Language Models

AdaLoop is a lightweight recurrent module that adaptively determines how many latent refinement steps are needed for audio–question pairs, allowing deeper reasoning only when necessary. It shares a transformer block that iterates over the audio representation guided by the question, with a learned halting mechanism that exits the loop once the representation is ready. Adding fewer than 3 % of the base model’s parameters, AdaLoop improves average accuracy by 2.9 to 3.8 points across three distinct models, especially on perception-heavy subtasks.

By Lee Seung-woo, Bowen Qi
arXiv AI
Oct 16

Qwen-Audio-Jev: A Unified Interface for Audio Decisions

Qwen-Audio-Jev extends Qwen-Audio-3.1-ASR-Flash-Next with a dedicated decision head that unifies selection, binary judgment, and scoring into three primitives: CHOICE, NOUL, and SCORE. The model delivers competitive accuracies and error rates across 18 labeled configurations, achieving 78.70% on MMAU-mini, 84.06% on MMSU, 6.50% equal error rate on VoxCeleb1-O, and a 0.1062 normalized MAE on NISQA. Compared to its predecessor, Qwen-Audio-Jev offers roughly a 20‑fold inference speedup, enabling efficient high‑throughput audio decision‑making for tasks such as large‑scale screening, speaker verification, and automated quality assessment.

By Ying Fang, Zhifu Gao, Qing Gu, Jiajun He, Kangqi Jing, Xiangang Li, Yu Pu, Wupeng Wang
arXiv AI
Sep 17

Look Less, Hear Better: Jointly Rewarded GRPO for Streaming ASR

The paper introduces AWED, a word‑level emission‑delay metric, and demonstrates that post‑training a delayed‑streaming model with a joint reward (GRPO) improves both accuracy and latency. Using a single operating point (τ=6 frames), the method outperforms both its supervised baseline and the Voxtral Realtime backbone across all lookahead budgets, reducing WER by up to 30.8% at 80 ms delay and lowering median AWED from 1.17 s to 1.04 s.

By Xiuwen Zheng
arXiv AI
Jun 4

Audio Interaction Model

arXiv:2606. 05121v1 Announce Type: cross Abstract: Audio is an inherently interactive modality, yet today's Large Audio Language Models (LALMs) are offline, and streaming audio models each handle only a single task such as streaming ASR or voice chatting.

By Zhifei Xie, Zihang Liu, Ze An, Xiaobin Hu, Yue Liao, Ziyang Ma, Dongchao Yang, Mingbao Lin, Deheng Ye, Shuicheng Yan, Chunyan Miao
arXiv Computation and Language
6d ago

Antalia-2 Mini: A 7.6-million-parameter Turkish text-to-speech model trained entirely on synthetic speech

Antalia-2 Mini is a single‑voice Turkish text‑to‑speech model with 7.6 million parameters that reads characters and outputs 48 kHz audio at 41× real time on an Apple M5 laptop CPU. It was trained solely on synthetic speech from the larger VoxCPM2 teacher, and fine‑tuning its vocoder on the model’s own mel spectrograms reduced Whisper’s word error rate from 3.89 % to 1.75 %. On the 495‑sentence Freya‑TR‑Eval benchmark, version 1.0.1 achieves a 1.09 % WER and 0.22 % character error rate, comparable to the EMA Lightning model in full‑band UTMOS scores.

By Sezgin Saygili, Emre Kaplaner, Oncel Ozgul, Fikri San Koktas