arXiv Computer Vision
Soundwich: Video Generation with Layered and Controllable Audio
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Machine Learning
Jul 1
Step-by-Step Video-to-Audio Synthesis via Negative Audio Guidance
arXiv:2506. 20995v4 Announce Type: replace-cross Abstract: We propose a step-by-step video-to-audio (V2A) generation method that provides finer control over the generation process and more realistic audio synthesis.
Hugging Face Trending Papers
Aug 6
Vorch-Omni: Multi-Task Orchestration of Sight and Sound
Recent advances in generative video modeling have enabled diverse generation, reference-based synthesis, extension, and editing, but existing approaches often rely on fragmented task-specific models. A general model must distinguish heterogeneous target, source, and reference signals to determine what to generate, preserve, or use as guidance, while reducing interference among tasks.
arXiv Computer Vision
Sep 3
Video Object Segmentation-Aware Audio Generation
The paper introduces a new task called video object segmentation‑aware audio generation, which conditions sound synthesis on object‑level segmentation maps. It presents SAGANet, a multimodal generative model that uses visual segmentation masks, video, and textual cues to produce controllable audio for musical instruments, offering fine‑grained, visually localized control. The authors also release the Segmented Music Solos dataset of instrument performance videos with segmentation information to support this task and demonstrate that SAGANet outperforms current state‑of‑the‑art methods in controllable, high‑fidelity Foley synthesis.
arXiv AI
Jul 24
OmniCustom: Sync Audio-Video Customization Via Joint Audio-Video Generation Model
arXiv:2602. 12304v5 Announce Type: replace-cross Abstract: Existing mainstream video customization methods focus on generating identity-consistent videos based on given reference images and textual prompts.
arXiv Computer Vision
4d ago
ControlFoley: Unified and Controllable Video-to-Audio Generation with Cross-Modal Conflict Handling
arXiv:2604.15086v3 Announce Type: replace-cross Abstract: Recent advances in video-to-audio (V2A) generation enable high-quality audio synthesis from visual content, yet achieving robust and fine-gra...
arXiv AI
4d ago
AVIO: Learning to Add and Remove Sounding Objects in Audiovisual Scenes
arXiv:2609.36503v1 Announce Type: new Abstract: Adding or removing a sounding object requires coordinated changes to visual content and sound while preserving the surrounding scene. Yet paired superv...