arXiv Machine Learning

Fine-grained Soundscape Control for Augmented Hearing

arXiv:2603. 00395v3 Announce Type: replace-cross Abstract: Hearables are becoming ubiquitous, yet their sound controls remain blunt: users can either enable global noise suppression or focus on a single target sound.

arXiv Computation and Language
Sep 21

I'll Keep an Ear Out: Teaching AudioLLMs Proactive Audio Assistance

arXiv:2609.21183v1 Announce Type: cross Abstract: Audio large language models (AudioLLMs) operate reactively, responding only when queried. We introduce proactive audio assistance, where an AudioLLM...

By Amit Kumar Singh Yadav, Ritvik Shrivastava, Xuan Zhang, Seungwhan Moon, Shashank Jain, Pinar Donmez, Babak Damavandi
arXiv AI
Sep 24

SsgCaps: A controlled dataset for the evaluation of sound scene generation algorithms

SsgCaps is a publicly available dataset of human-engineered sound scenes, each paired with a precisely structured prompt that guides the sampling process. The prompts are drawn from a predefined action-based typology, enabling extensive yet plausible sampling. A comparative quantitative analysis shows only small differences between the open and private versions, supporting the recommendation of the open version for benchmarking sound scene generation algorithms.

By Modan Tailleur (LS2N), Junwon Lee (LS2N), Laurie M Heller (LS2N), Mathieu Lagrange (LS2N), Keunwoo Choi, Brian McFee, Keisuke Imoto, Yuki Okamoto
arXiv AI
Jun 4

Audio Interaction Model

arXiv:2606. 05121v1 Announce Type: cross Abstract: Audio is an inherently interactive modality, yet today's Large Audio Language Models (LALMs) are offline, and streaming audio models each handle only a single task such as streaming ASR or voice chatting.

By Zhifei Xie, Zihang Liu, Ze An, Xiaobin Hu, Yue Liao, Ziyang Ma, Dongchao Yang, Mingbao Lin, Deheng Ye, Shuicheng Yan, Chunyan Miao
arXiv AI
Sep 7

SCAPES: Semantically Conditioned Autoregressive Prior for Environmental Sounds

SCAPES is a lightweight, resource‑efficient generative model that synthesizes high‑fidelity environmental sounds with high‑level semantic control. It operates on the continuous latent manifold of a neural audio codec, using a segmentation strategy and a Continuous Normalizing Flow to model latent trajectories. A 36‑million‑parameter instance can be trained on limited, uncurated data with a single consumer‑grade GPU, achieving convergence in roughly twice the source audio duration and enabling smooth semantic interpolation.

By Esteban Guti\'errez, Lonce Wyse, Frederic Font, Xavier Serra
arXiv Computer Vision
Aug 28

StreamAV-Bench: A Comprehensive Benchmark for Streaming Audio-Video Generation

StreamAV-Bench is the first comprehensive benchmark designed for streaming audio‑video generation, addressing the limitations of existing benchmarks that focus on completed sequences. It introduces a unified evaluation framework with a progressive track for instruction adherence and long‑horizon stability, and an interactive track for responsive interaction and state retention. The benchmark includes 32 fine‑grained, expert‑verified evaluation cases and evaluates 13 representative systems, revealing temporal drift in progressive generation and responsiveness bottlenecks in interactive control.

By Kaiqi Liu, Haoxuan Zeng, Jingqi Liu, Jiacong Fang, Ziqi Cai, Yunyao Mao, Henglin Liu, Yu Sheng, Shuchen Weng, Boxin Shi