BinauralVAE is an open‑source pipeline that reconstructs spatial audio using various Variational Autoencoder architectures, including complex‑valued variants, to learn latent representations of binaural signals. The project builds on realistic acoustic data from a simulated robot navigating an environment, providing a foundation for audio‑centric world models. It aims to map the causal link between navigational actions and their acoustic outcomes, positioning sound as a complementary modality for spatial awareness.
By Luis Vitor Zerkowski, Luiz Velho
arXiv:2509. 15210v2 Announce Type: replace-cross Abstract: Realistic sound simulation plays a critical role in many applications.
By Chen Si, Qianyi Wu, Chaitanya Amballa, Romit Roy Choudhury
arXiv:2607. 04848v1 Announce Type: cross Abstract: While audio deepfake detection has advanced significantly, representative detectors show limited generalization to synthetic sound effects.
By Linxi Li, Yuncong Yu, Qianwei Guo, Liwei Jin, Yechen Wang, Carsten Maple
arXiv:2607. 16736v1 Announce Type: cross Abstract: This paper presents RealDESED, a real-world domestic sound event detection (SED) benchmark comprising 5,710 audio recordings collected by 652 participants in their homes.
By Florian Schmid, Paul Primus, Alexander Fichtinger, Tara Jadidi, Tobias Morocutti, Gerhard Widmer
arXiv:2606. 14141v1 Announce Type: cross Abstract: Sound events are entities with semantic identities, locations, and trajectories, but current audio-language models usually reason about clips as global event content.
By Oh Hyun-Bin, Kazuki Shimada, Yuhta Takida, Kim Sung-Bin, Toshimitsu Uesaka, Takashi Shibuya, Kyeongyoon Lee, Tae-Hyun Oh, Yuki Mitsufuji
arXiv:2402. 01591v4 Announce Type: replace-cross Abstract: Spatial sound reasoning is a fundamental human skill, enabling us to navigate and interpret our surroundings based on sound.
By Zhisheng Zheng, Puyuan Peng, Ziyang Ma, Xie Chen, Eunsol Choi, David Harwath
NOPE-HYPE is a structured training workflow that integrates a controllable environment simulator, a coverage‑optimal reduction of Power Spectral Density templates, and a concise hyperparameter search over simulator settings. The authors demonstrate that noise generated by the simulator can match the performance of balanced real‑noise training for Whisper and SeamlessM4T models. They also provide principled environment prototype sets and practical default simulator configurations derived from a 27‑run hyperparameter sweep.
By Niramay M. Patel, Bibek Behera, Raksha Sharma
arXiv:2603. 23667v2 Announce Type: replace-cross Abstract: We introduce Echoes, a new dataset for music deepfake detection designed for training and benchmarking detectors under realistic and provider-diverse conditions.
By Octavian Pascu, Dan Oneata, Horia Cucu, Nicolas M. Muller
PhysWave is a physics-guided latent diffusion model designed for controllable text-to-First-Order Ambisonics (FOA) audio generation. It integrates natural-language and trajectory control via a shared waypoint-caption representation and incorporates two differentiable acoustic priors—spherical-harmonic direction consistency and inverse-square distance consistency—into diffusion training. The authors also introduce a 300K-clip FOA dataset and demonstrate that these priors improve spatial consistency while preserving audio quality, with potential use as inference-time guidance for training-free refinement.
By Lingfeng Yao, Chenpei Huang, Xingke Yang, Ziye Geng, Changqing Luo, Hao Wang, Jiang Liu, Miao Pan
arXiv:2502. 16584v2 Announce Type: replace-cross Abstract: Recent advancements in audio tokenization have significantly enhanced the integration of audio capabilities into large language models (LLMs).
By Liumeng Xue, Ziya Zhou, Jiahao Pan, Zixuan Li, Shuai Fan, Yinghao Ma, Sitong Cheng, Dongchao Yang, Haohan Guo, Yujia Xiao, Xinsheng Wang, Zixuan Shen, Chuanbo Zhu, Xinshen Zhang, Tianchi Liu, Ruibin Yuan, Zeyue Tian, Haohe Liu, Xingjian Du, Emmanouil Benetos, Ge Zhang, Yike Guo, Wei Xue
Robust speech-to-text translation systems should perform reliably across diverse acoustic conditions, yet practical pipelines lack controllable tools for systematic environment exploration. Large spee...