The paper introduces a lightweight technique to steer the pitch content of audio generated by the Stable Audio Open diffusion model. A small convolutional probe (~125k parameters) is trained to decode frame‑level pitch‑class activations from the model’s latent space using paired audio and MIDI data. During inference, the frozen probe acts as a differentiable loss, guiding generation toward a user‑specified pitch‑class sequence without retraining the base model, and improves melodic coherence by 2.4× over the unguided baseline.
arXiv:2511. 05350v3 Announce Type: replace-cross Abstract: We argue that training autoencoders to reconstruct inputs from noised versions of their encodings, when combined with perceptually motivated losses, yields encodings that are structured according to a perceptual hierarchy.
arXiv:2606. 07015v1 Announce Type: cross Abstract: While song generation and singing voice conversion (SVC) have evolved significantly, they have long been developed isolated: the former lacks zero-shot speaker cloning, while the latter overlooks vocal-accompaniment synergy.
arXiv:2608. 04378v1 Announce Type: cross Abstract: Collaborative music agents need internal representations rich enough to support both understanding and generation, yet flexible enough for a workflow where the human retains agency.
arXiv:2606. 02638v1 Announce Type: cross Abstract: Recent advances in neural song generation have enabled high-quality synthesis from lyrics and global textual prompts.
PhysWave is a physics-guided latent diffusion model designed for controllable text-to-First-Order Ambisonics (FOA) audio generation. It integrates natural-language and trajectory control via a shared waypoint-caption representation and incorporates two differentiable acoustic priors—spherical-harmonic direction consistency and inverse-square distance consistency—into diffusion training. The authors also introduce a 300K-clip FOA dataset and demonstrate that these priors improve spatial consistency while preserving audio quality, with potential use as inference-time guidance for training-free refinement.
arXiv:2606. 24307v1 Announce Type: cross Abstract: Interactive music and live performance relies on real-time human expression, but modern generative music AI remains largely absent from this domain due to its prohibitive inference latency and offline rendering paradigm.
arXiv:2606. 03169v1 Announce Type: cross Abstract: Recent song generation systems can synthesize realistic audio, yet generating complete songs remains challenging for two reasons.
Zero-shot text-guided editing of real-world music recordings requires balancing semantic modification with faithful preservation of the original musical structure. Although recent diffusion transformers trained with rectified flow have achieved remarkable success in text-to-music generation, extending them to edit existing recordings remains challenging because editing requires accurate deterministic inversion, reliable structural preservation, and numerically stable integration throughout the inversion and generation processes.
DuoTok is a source‑aware dual‑track music tokenizer designed for vocal‑accompaniment generation. It first learns a semantic audio representation via self‑supervised pretraining, then refines source‑aware structure with feature‑replacement noise and multi‑task supervision (spectral reconstruction, source separation regularization, and an ASR head for lyric alignment). The encoder is frozen and hard‑routed codebooks for vocals and accompaniment are learned, while a diffusion decoder restores fine acoustic detail from the discrete tokens, achieving a favorable predictability‑fidelity trade‑off at ultra‑low bitrate across public benchmarks.
arXiv:2506. 14293v4 Announce Type: replace-cross Abstract: We present Sleeping-DISCO 9M, a large-scale pre-training dataset for music and song.
arXiv:2607. 17526v1 Announce Type: cross Abstract: Zero-shot text-guided editing of real-world music recordings requires balancing semantic modification with faithful preservation of the original musical structure.
arXiv:2608.30125v1 Announce Type: cross Abstract: Current video-to-music (V2M) models lack semantic control and fail to penalize instruction violations, largely due to their reliance on reconstructio...