arXiv AI
VIBE: Video Instruction-aligned Background music gEneration
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
4d ago
Self-Representation Alignment in Diffusion Models for Video Background Music Generation
arXiv:2610.15575v1 Announce Type: new Abstract: Diffusion models have been widely used for various generative tasks. However, recent studies have demonstrated that the internal representations learne...
arXiv Computer Vision
Sep 25
AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation
AV‑GRPO introduces a modality‑anchored diffusion reinforcement learning framework for joint audio‑video generation, addressing limitations in fidelity, text‑modality alignment, and cross‑modal synchronization. It decouples learning signals through modality‑anchored rollouts, employs trajectory‑locked frozen‑tower optimization to reduce computational cost, and adapts objectives to each modality’s dynamics. The accompanying 5DAV dataset provides difficulty‑controllable, decoupled training samples, and experiments on JavisBench and VABench show AV‑GRPO surpasses LTX‑2.3 in generation quality, semantic alignment, and synchronization.
Hugging Face Trending Papers
4d ago
AVC-Reward: Event-Centric Audio-Video Consistency Evaluation
AVC-Reward introduces an event‑centric framework for evaluating audio‑video consistency in joint generation models. It separates audio and visual events, assesses them independently, cross‑validates cross‑modal relations, and aggregates results into a three‑level metric covering global narrative harmony, event‑level synchronization, and fine‑grained attribute alignment. The resulting AVC‑Reward model aligns closely with human judgments, offers interpretable failure localization, and enhances consistency when used as a reinforcement‑learning signal.
arXiv AI
3d ago
AVC-Reward: Event-Centric Audio-Video Consistency Evaluation
AVC-Reward introduces an event‑centric framework for evaluating audio‑video consistency, breaking down scenes into individual audio and visual events and assessing them at three levels: global narrative harmony, event‑level temporal synchronization, and fine‑grained attribute alignment. The metric, instantiated as a trainable reward model, is validated against human judgments using the AVC‑Benchmark, which contains diverse positive and negative samples from various generation models. AVC‑Reward outperforms existing automatic methods, offers interpretable failure localization, and enhances consistency when used as a reinforcement‑learning signal.
arXiv AI
Jun 2
JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions
arXiv:2606. 01703v1 Announce Type: cross Abstract: We address the challenge of generating high-fidelity, long-form soundtracks that remain coherent across scene transitions.
arXiv Computer Vision
Aug 21
VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation
arXiv:2608. 18607v2 Announce Type: replace Abstract: Using reinforcement learning to post-train joint video-audio generation models requires a reward signal.