arXiv Computer Vision
Edit in Sync: Unified End-to-End Audio-Visual Editing for Human-Centric Videos
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
2d ago
Vorch-IR: Long-Form Unified Multimodal Identity Replacement Video Generation
arXiv:2608.05648v2 Announce Type: replace Abstract: Video identity replacement seeks to transfer the identities of one or more subjects while preserving the motion, expressions, and temporal structur...
arXiv Computer Vision
Oct 12
AV-FlowEdit: Training-Free Prompt-Driven Audio-Visual Editing with Event-Aware Spectral Trajectory Preservation
AV-FlowEdit is a training‑free, inversion‑free framework that enables prompt‑driven audio‑visual editing using joint audio‑video diffusion models. It leverages source‑to‑target flow differences to guide edits while preserving non‑target structure and cross‑modal correspondence. The method includes a Cross‑Modal Event‑Aware Spectral Trajectory Preservation module that suppresses drift by combining flow saliency, aligned audio activity, multiscale low‑frequency displacement, and diffusion‑stage scheduling, achieving strong prompt alignment, temporal consistency, and competitive audio quality.
arXiv Computer Vision
Sep 23
Vorch-Human: Unified Multi-Task Human-Centric Generation via Long-Horizon Continuation
Vorch-Human is a unified framework for human‑centric audio‑visual generation that handles multiple tasks—animating a person from speech, jointly generating speech and video from a voice reference, and synthesizing a scene from paired appearance and voice references—using a single dual‑stream audio‑video diffusion transformer. The model incorporates clean condition‑audio and condition‑video tokens, per‑token task embeddings, temporal position types, condition masks, and a shared multimodal prompt encoder to express diverse inputs such as driving speech, timbre examples, first frames, and subject images. A two‑level data pipeline supplies the necessary supervision by extracting speech, appearance, and timbre annotations from clips and linking consistent identity and outfit references across videos, while a frozen‑prefix recurrence enables long‑form audio‑driven generation with reduced boundary discontinuity and identity drift.
arXiv AI
Sep 30
AVIO: Learning to Add and Remove Sounding Objects in Audiovisual Scenes
arXiv:2609.36503v1 Announce Type: new Abstract: Adding or removing a sounding object requires coordinated changes to visual content and sound while preserving the surrounding scene. Yet paired superv...
arXiv Computer Vision
Sep 7
AVENUE: Audio-Video EditiNg Understanding and Evaluation
AVENUE is a new benchmark and evaluation framework for audio‑video editing that includes 1,291 source clips and 7,957 editing instructions covering audio‑targeted, video‑targeted, and coupled edits. It introduces a sample‑specific, modality‑aware evaluation that specifies the intended change and the content that must remain unchanged. The study applies this framework to joint, sequential, and separate editing models, revealing that existing models often alter unintended modalities, highlighting a key challenge in controllable AV editing.
arXiv Computer Vision
Oct 2
Soundwich: Video Generation with Layered and Controllable Audio
arXiv:2610.00691v1 Announce Type: new Abstract: Recent joint audio-video generative models can synthesize realistic videos with synchronized sound, but typically generate audio as a single mixed trac...