arXiv Computer Vision
Tora3: Trajectory-Guided Audio-Video Generation with Physical Coherence
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Oct 1
Here the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generation
arXiv:2609.38748v1 Announce Type: new Abstract: Recent joint video-audio generation models have achieved strong semantic correspondence and temporal synchronization. However, applications such as AR/...
arXiv Computer Vision
5d ago
Hear the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generation
The paper introduces StereoBind, a framework that links visual source motion to stereo audio generation, addressing the need for dynamic spatial correspondence in immersive applications like AR/VR and gaming. It employs three mechanisms—Visual Motion Binding, Spatial Track Encoder, and Residual Track RoPE—to coordinate visual motion with stereo sound. The authors also present StereoWorld-29K, a large-scale dataset with paired motion tracks, and StereoWorldBench for evaluating audiovisual spatial consistency, demonstrating that StereoBind improves spatial alignment while maintaining overall quality.
arXiv AI
Oct 7
WorldSonus: Bringing Sound to Worlds
arXiv:2610.08760v1 Announce Type: cross Abstract: Recent advances in world models have enabled increasingly realistic visual synthesis. However, these generated environments remain largely silent. Br...
arXiv Computer Vision
Sep 22
MoSAT: Human Motion Generation from Spatial Audio and Textual Description
arXiv:2609.23797v1 Announce Type: cross Abstract: Human motion is shaped by both external acoustic events and behavioral intent: spatial audio conveys environmental cues that elicit or guide a respon...
arXiv Computer Vision
Sep 24
MotionSpec: Spectral Trajectory Supervision for Motion-Consistent Video Generation
MotionSpec introduces a motion supervision framework for text-to-video generation that focuses on Spectral Trajectory Consistency (STC). STC builds dense anchor-relative motion trajectories, transforms them into spectral volumes, and aligns their amplitude and phase with target trajectories to constrain motion strength and temporal organization. The framework also adds Local Flow Consistency (LFC) to stabilize local motion transitions, resulting in improved motion consistency, temporal coherence, and plausibility while maintaining visual fidelity.
arXiv Computer Vision
2d ago
AV-FlowEdit: Training-Free Prompt-Driven Audio-Visual Editing with Event-Aware Spectral Trajectory Preservation
AV-FlowEdit is a training‑free, inversion‑free framework that enables prompt‑driven audio‑visual editing using joint audio‑video diffusion models. It leverages source‑to‑target flow differences to guide edits while preserving non‑target structure and cross‑modal correspondence. The method includes a Cross‑Modal Event‑Aware Spectral Trajectory Preservation module that suppresses drift by combining flow saliency, aligned audio activity, multiscale low‑frequency displacement, and diffusion‑stage scheduling, achieving strong prompt alignment, temporal consistency, and competitive audio quality.