arXiv AI

PhysicsSliders: Learning Physical Controls for Audio Foley Generation

arXiv AI
Sep 17

Dreaming the Sound of Contact: Leveraging Video and Audio Generation for Zero-Shot Force-Aware Manipulation and Data Generation

The paper introduces a pipeline that combines generated video and audio to produce force-aware manipulation trajectories for a Franka Panda robot. By using the loudness of contact sounds to shape a bounded, time-varying desired-force profile, the system can execute tasks that require precise contact forces, outperforming kinematic-only baselines. The approach also serves as a data generation engine for training closed-loop policies.

By Guanhua Ji, Tianyu Li, Dayoon Suh, Yuqian Zhang, Boyan Zhang, Nadia Figueroa
arXiv AI
Oct 15

PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation

arXiv:2512.23994v5 Announce Type: replace-cross Abstract: Text-to-audio-video (T2AV) generation is central to applications such as filmmaking and world modeling. However, current models often fail to...

By Tianxin Xie, Wentao Lei, Kai Jiang, Guanjie Huang, Pengfei Zhang, Chunhui Zhang, Fengji Ma, Haoyu He, Han Zhang, Jiangshan He, Jinting Wang, Linghan Fang, Lufei Gao, Orkesh Ablet, Peihua Zhang, Ruolin Hu, Shengyu Li, Weilin Lin, Xiaoyang Feng, Xinyue Yang, Yan Rong, Yanyun Wang, Zihang Shao, Zelin Zhao, Chenxing Li, Shan Yang, Wenfu Wang, Meng Yu, Dong Yu, Li Liu
arXiv AI
Sep 24

SsgCaps: A controlled dataset for the evaluation of sound scene generation algorithms

SsgCaps is a publicly available dataset of human-engineered sound scenes, each paired with a precisely structured prompt that guides the sampling process. The prompts are drawn from a predefined action-based typology, enabling extensive yet plausible sampling. A comparative quantitative analysis shows only small differences between the open and private versions, supporting the recommendation of the open version for benchmarking sound scene generation algorithms.

By Modan Tailleur (LS2N), Junwon Lee (LS2N), Laurie M Heller (LS2N), Mathieu Lagrange (LS2N), Keunwoo Choi, Brian McFee, Keisuke Imoto, Yuki Okamoto
arXiv Machine Learning
Oct 22

Decodability Is Not Controllability: Local and Anisotropic Speaker Control in Continuous-Latent TTS

The paper demonstrates that high linear decodability in continuous‑latent TTS does not guarantee causal controllability of speaker identity. In VoxCPM2, probes at the final residual acoustic language model layer show strong accuracy, yet aligned edits fail to produce target‑dominant dyads, whereas early‑position edits succeed. Cross‑architecture experiments reveal that backbone edits shift the source voice without attracting the target, while x‑vector ablation eliminates source similarity, underscoring the need for geometric specificity and a computational path that embeds the intended speaker condition into synthesis.

By Yuanzhong Zheng, Wei Huang, Ming Ma, Kailun Jin, Youhai Jiang
arXiv Machine Learning
5d ago

TAPE: Learning General Audio Effect Representations

TAPE (Transformations of Audio Parameter Embeddings) is a system that learns to infer the signal‑processing chain applied to a reference audio recording and then applies the same chain to new material. It uses an encoder to map the processed reference to an embedding that a differentiable DSP decoder translates into editable parameters for equalization, compression, waveshaping, a residual network, and reverberation. Compared to prior methods, TAPE achieves lower spectral error while being 275× faster than exhaustive search and can run causally in real time on 128‑sample buffers.

By Phillip Long, Yongyi Zang, Qihui Yang, Christodoulos Benetatos, Junfei Liu, Randal Leistikow, Julian McAuley, Perry R. Cook