arXiv AI
Self-Representation Alignment in Diffusion Models for Video Background Music Generation
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 1
VIBE: Video Instruction-aligned Background music gEneration
arXiv:2608.30125v1 Announce Type: cross Abstract: Current video-to-music (V2M) models lack semantic control and fail to penalize instruction violations, largely due to their reliance on reconstructio...
arXiv Machine Learning
Jul 9
Gen4U: Unifying Video Generation and Understanding via Diffusion
arXiv:2607. 06856v1 Announce Type: cross Abstract: Prior work suggests that diffusion representations capture low-level geometry but struggle with high-level semantics.
arXiv AI
Jul 8
Perceptually Aligning Representations of Music via Noise-Augmented Autoencoders
arXiv:2511. 05350v3 Announce Type: replace-cross Abstract: We argue that training autoencoders to reconstruct inputs from noised versions of their encodings, when combined with perceptually motivated losses, yields encodings that are structured according to a perceptual hierarchy.
arXiv Machine Learning
Aug 17
SALSA-V: Shortcut-Augmented Long-form Synchronized Audio from Videos
arXiv:2510. 02916v2 Announce Type: replace-cross Abstract: We propose SALSA-V, a multimodal video-to-audio generation model capable of synthesizing highly synchronized, high-fidelity long-form audio from silent video content.
arXiv AI
Sep 7
Pitch-class Steering for Diffusion-based Music Generation via Latent-space Probes
The paper introduces a lightweight technique to steer the pitch content of audio generated by the Stable Audio Open diffusion model. A small convolutional probe (~125k parameters) is trained to decode frame‑level pitch‑class activations from the model’s latent space using paired audio and MIDI data. During inference, the frozen probe acts as a differentiable loss, guiding generation toward a user‑specified pitch‑class sequence without retraining the base model, and improves melodic coherence by 2.4× over the unguided baseline.
arXiv AI
4d ago
Fast Diffusion-GAN Model for Generating Symbolic Music with Emotions
arXiv:2310.14040v2 Announce Type: replace-cross Abstract: Diffusion models have shown promising results for a wide range of generative tasks with continuous data, such as image and audio synthesis. H...