← Back to all news
arXiv AI By Claudia Montero-Ram\'irez, Alba Mart\'inez-Serrano, Carmen Pel\'aez-Moreno

Hierarchical Interpretable Multitask Audio Representation Learning: An Analysis of Event-Scene Relationships through the Information Bottleneck

Read the original on arXiv AI →

The paper proposes a hierarchical conditioning approach for joint Acoustic Scene Classification and Audio Tagging, predicting scene categories directly from intermediate event probabilities. Extensive experiments on three backbones and varied loss‑weighting settings show that this method matches or surpasses traditional multi‑head models while enabling synchronized dual‑task learning for certain beta values. An interpretability framework combining information bottleneck analysis, manifold geometry, and Formal Concept Analysis on TF‑IDF event representations demonstrates that hierarchical conditioning reduces conditional entropy, increases mutual information, and improves latent cluster separability, yielding a more structured and semantically faithful feature space.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

Hugging Face Trending Papers
Jun 24

From Sounds to Scenes: A Benchmark for Evaluating Context-Aware Auditory Scene Understanding in Large Audio Language Models

Recent Large Audio Language Models (LALMs) have achieved remarkable progress in audio perceptual tasks across individual acoustic layers, including speech, sound, and music. However, existing benchmarks predominantly evaluate these layers in isolation, overlooking the complex contextual relationships that arise when multiple acoustic sources co-occur in real-world auditory scenes.

llmsnlpbenchmarks
More like this →
arXiv AI
Oct 1

UniAE-MoE: A Unified Audio Encoder via Mixture of Experts

UniAE-MoE is a unified audio encoder that uses a Mixture-of-Experts architecture to model cross‑domain audio representations. It integrates encoder components from Qwen2‑Audio and Audio‑Flamingo 3, enhances them with SwiGLU and shared experts, and applies a two‑stage instruction‑tuning strategy along with task‑specific data scaling. The model achieves state‑of‑the‑art results on the XARES‑LLM benchmark (0.802) and tops the Interspeech 2026 Audio Encoder Capability Challenge, demonstrating strong generalization across speech, music, and general audio tasks.

By Shengbo Cai, Zhisheng Zhang, Zichao Nie, Jing Peng, Jingran Xie, Zhiyong Wu
llmsbenchmarks
More like this →
arXiv AI
Oct 16

AVC-Reward: Event-Centric Audio-Video Consistency Evaluation

AVC-Reward introduces an event‑centric framework for evaluating audio‑video consistency, breaking down scenes into individual audio and visual events and assessing them at three levels: global narrative harmony, event‑level temporal synchronization, and fine‑grained attribute alignment. The metric, instantiated as a trainable reward model, is validated against human judgments using the AVC‑Benchmark, which contains diverse positive and negative samples from various generation models. AVC‑Reward outperforms existing automatic methods, offers interpretable failure localization, and enhances consistency when used as a reinforcement‑learning signal.

By Yuruo Wu, Kaihang Pan, Kai Shen, Jiacheng Li, Shuyuan Tu, Jisheng Dang, Li Tuoyu, Juncheng Li, Siliang Tang
diffusionreinforcement-learningmultimodalbenchmarkssafety
More like this →