← Back to all news
arXiv Machine Learning By Francesco Brigante, Luca Cerovaz, Davide Marincione, Giorgio Strano, Luca Zhou, Emanuele Rodol\`a, Michele Mancusi

SAGE: Semantic Audio Generative Encoder

Read the original on arXiv Machine Learning →

SAGE, a 105‑million‑parameter semantic audio generative encoder, compresses music waveforms into compact latent representations by distilling embeddings from a pretrained audio‑text model. It achieves a three‑way balance—high reconstruction quality, rich semantic latent structure, and fast inference—running three times faster than Stable Audio Open while matching or surpassing larger models in listening‑test quality and objective metrics. Across all nineteen latent‑semantics probing tasks, SAGE sets new state‑of‑the‑art performance both within and outside its training domain.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Oct 9

How Much Audio Is Left In An Embedding? An Inversion Audit Of Audio Encoders

The paper investigates how much original audio information remains in frozen representations from various pretrained audio encoders by attempting to reconstruct five‑second, 44.1‑kHz stereo music clips. Using a shared Stable Audio Open latent diffusion decoder, the authors reconstruct audio from embeddings produced by supervised classifiers (VGGish, ConvNeXt), an audio‑text contrastive model (CLAP), and a waveform‑reconstruction model (EnCodec). Results on the Million Song Dataset reveal that reconstructability varies across encoder families and that embeddings exposing finer temporal or spectral detail enable better recovery of source content, even for compressed, task‑oriented embeddings.

By Marios Glytsos, Brian McFee
ragdiffusion
More like this →
arXiv AI
Jul 7

Unified Audio Intelligence Without Regressing on Text Intelligence

arXiv:2607. 05196v1 Announce Type: cross Abstract: Audio intelligence involves understanding, reasoning about, and generating both audio and speech.

By Zhifeng Kong, Sang-gil Lee, Jaehyeon Kim, Boxin Wang, Zihan Liu, Sungwon Kim, Yang Chen, Arushi Goel, Rajarshi Roy, Wenliang Dai, Zhuolin Yang, Yangyi Chen, Dongfu Jiang, Sreyan Ghosh, Tuomas Rintamaki, Andrew Tao, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
llmsragagentsreinforcement-learningefficiencymultimodalbenchmarkssafety
More like this →
arXiv Machine Learning
Oct 7

AdaLoop: Adaptive-Depth Latent Reasoning for Audio Language Models

AdaLoop is a lightweight recurrent module that adaptively determines how many latent refinement steps are needed for audio–question pairs, allowing deeper reasoning only when necessary. It shares a transformer block that iterates over the audio representation guided by the question, with a learned halting mechanism that exits the loop once the representation is ready. Adding fewer than 3 % of the base model’s parameters, AdaLoop improves average accuracy by 2.9 to 3.8 points across three distinct models, especially on perception-heavy subtasks.

By Lee Seung-woo, Bowen Qi
llms
More like this →
arXiv AI
Sep 7

SCAPES: Semantically Conditioned Autoregressive Prior for Environmental Sounds

SCAPES is a lightweight, resource‑efficient generative model that synthesizes high‑fidelity environmental sounds with high‑level semantic control. It operates on the continuous latent manifold of a neural audio codec, using a segmentation strategy and a Continuous Normalizing Flow to model latent trajectories. A 36‑million‑parameter instance can be trained on limited, uncurated data with a single consumer‑grade GPU, achieving convergence in roughly twice the source audio duration and enabling smooth semantic interpolation.

By Esteban Guti\'errez, Lonce Wyse, Frederic Font, Xavier Serra
diffusioncomputer-visionnlp
More like this →