arXiv Machine Learning By Phillip Long, Yongyi Zang, Qihui Yang, Christodoulos Benetatos, Junfei Liu, Randal Leistikow, Julian McAuley, Perry R. Cook

TAPE: Learning General Audio Effect Representations

Read the original on arXiv Machine Learning →

TAPE (Transformations of Audio Parameter Embeddings) is a system that learns to infer the signal‑processing chain applied to a reference audio recording and then applies the same chain to new material. It uses an encoder to map the processed reference to an embedding that a differentiable DSP decoder translates into editable parameters for equalization, compression, waveshaping, a residual network, and reverberation. Compared to prior methods, TAPE achieves lower spectral error while being 275× faster than exhaustive search and can run causally in real time on 128‑sample buffers.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Oct 9

How Much Audio Is Left In An Embedding? An Inversion Audit Of Audio Encoders

The paper investigates how much original audio information remains in frozen representations from various pretrained audio encoders by attempting to reconstruct five‑second, 44.1‑kHz stereo music clips. Using a shared Stable Audio Open latent diffusion decoder, the authors reconstruct audio from embeddings produced by supervised classifiers (VGGish, ConvNeXt), an audio‑text contrastive model (CLAP), and a waveform‑reconstruction model (EnCodec). Results on the Million Song Dataset reveal that reconstructability varies across encoder families and that embeddings exposing finer temporal or spectral detail enable better recovery of source content, even for compressed, task‑oriented embeddings.

By Marios Glytsos, Brian McFee
arXiv AI
Sep 17

CPR: Combining global composing, local performing and full-sequence refining in piano rendering with continuous autoregressive modelling

The paper introduces CPR, a piano rendering framework that combines continuous autoregressive modeling with local flow matching and full‑sequence refinement. It predicts continuous hidden states, generates 24 kHz acoustic latents, and upsamples to 48 kHz, while new techniques BREPA and MT‑RoPE enhance musical semantics and cross‑modal alignment.

By Chong Jing, Junan Zhang, Zhizheng Wu
arXiv AI
Aug 18

Adding Voice Cloning to Text-to-Audio-Video Models with a Single Zero-Initialised Layer

arXiv:2608. 15690v1 Announce Type: cross Abstract: Text-to-audio-video (T2AV) generation models produce a video and its soundtrack from a textual description, but offer no control over whose voice speaks in the output.

By Ivan Mikheev, Viacheslav Vasilev, Anna Dmitrienko, Alexey Letunovskiy, Ivan Kirillov, Kirill Chernyshev, Denis Dimitrov