arXiv:2609.38748v1 Announce Type: new Abstract: Recent joint video-audio generation models have achieved strong semantic correspondence and temporal synchronization. However, applications such as AR/...
The paper introduces StereoBind, a framework that links visual source motion to stereo audio generation, addressing the need for dynamic spatial correspondence in immersive applications like AR/VR and gaming. It employs three mechanisms—Visual Motion Binding, Spatial Track Encoder, and Residual Track RoPE—to coordinate visual motion with stereo sound. The authors also present StereoWorld-29K, a large-scale dataset with paired motion tracks, and StereoWorldBench for evaluating audiovisual spatial consistency, demonstrating that StereoBind improves spatial alignment while maintaining overall quality.
arXiv:2610.08760v1 Announce Type: cross Abstract: Recent advances in world models have enabled increasingly realistic visual synthesis. However, these generated environments remain largely silent. Br...
arXiv:2609.23797v1 Announce Type: cross Abstract: Human motion is shaped by both external acoustic events and behavioral intent: spatial audio conveys environmental cues that elicit or guide a respon...
MotionSpec introduces a motion supervision framework for text-to-video generation that focuses on Spectral Trajectory Consistency (STC). STC builds dense anchor-relative motion trajectories, transforms them into spectral volumes, and aligns their amplitude and phase with target trajectories to constrain motion strength and temporal organization. The framework also adds Local Flow Consistency (LFC) to stabilize local motion transitions, resulting in improved motion consistency, temporal coherence, and plausibility while maintaining visual fidelity.
AV-FlowEdit is a training‑free, inversion‑free framework that enables prompt‑driven audio‑visual editing using joint audio‑video diffusion models. It leverages source‑to‑target flow differences to guide edits while preserving non‑target structure and cross‑modal correspondence. The method includes a Cross‑Modal Event‑Aware Spectral Trajectory Preservation module that suppresses drift by combining flow saliency, aligned audio activity, multiscale low‑frequency displacement, and diffusion‑stage scheduling, achieving strong prompt alignment, temporal consistency, and competitive audio quality.
InteractGesture is a model‑agnostic, inference‑time method that enables fine‑grained spatial control of individual joints in continuous streaming co‑speech gesture generation. It guides diffusion sampler latent estimates through a differentiable RVQ‑VAE decoder, backpropagating spatial control gradients to adjust motion latents during sampling. To address chunk‑wise dependency issues in streaming generation, the method introduces Progressive Chunk Guidance, a chunk‑window strategy that keeps an active set of editable chunk latents with staggered delays, allowing spatial constraints to propagate gradients backward across chunk boundaries and reducing boundary inconsistencies.
The paper introduces a pipeline that combines generated video and audio to produce force-aware manipulation trajectories for a Franka Panda robot. By using the loudness of contact sounds to shape a bounded, time-varying desired-force profile, the system can execute tasks that require precise contact forces, outperforming kinematic-only baselines. The approach also serves as a data generation engine for training closed-loop policies.
The paper introduces MOCO, a diffusion-based framework that generates 3D avatar motions from concurrent multimodal inputs such as speech audio, text descriptions, and trajectory data. MOCO decouples motion generation by independently producing modality-specific motions at each denoising step and then assembling them according to spatial rules, iteratively refining the combined motion. This approach yields coherent, lifelike, and synchronized movements, outperforming existing baselines on a multimodal benchmark.
arXiv:2608.20770v1 Announce Type: new Abstract: Modern AI video generation models can produce videos with high visual fidelity and seemingly smooth temporal transitions. However, visual realism does...
The paper introduces a Prior‑Guided Residual Flow Matching framework for 3D multi‑person motion prediction. It uses a Deterministic Coarse Prior to anchor kinematics and a Dynamic Cross‑Interaction mechanism to synchronize inter‑agent message passing during integration, thereby improving structural consistency and social context extraction. A decoupled joint‑motion architecture with bidirectional fusion further preserves fine‑grained kinematic coherence, achieving state‑of‑the‑art accuracy on several datasets.
arXiv:2608. 16285v1 Announce Type: cross Abstract: Audio-Visual Segmentation (AVS) is a fundamental task in multimodal perception that performs pixel-level segmentation of sounding objects in videos by leveraging both visual and audio cues.