Hear the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generation

The paper introduces StereoBind, a framework that links visual source motion to stereo audio generation, addressing the need for dynamic spatial correspondence in immersive applications like AR/VR and gaming. It employs three mechanisms—Visual Motion Binding, Spatial Track Encoder, and Residual Track RoPE—to coordinate visual motion with stereo sound. The authors also present StereoWorld-29K, a large-scale dataset with paired motion tracks, and StereoWorldBench for evaluating audiovisual spatial consistency, demonstrating that StereoBind improves spatial alignment while maintaining overall quality.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

• safety