arXiv:2610.15575v1 Announce Type: new Abstract: Diffusion models have been widely used for various generative tasks. However, recent studies have demonstrated that the internal representations learne...
AV‑GRPO introduces a modality‑anchored diffusion reinforcement learning framework for joint audio‑video generation, addressing limitations in fidelity, text‑modality alignment, and cross‑modal synchronization. It decouples learning signals through modality‑anchored rollouts, employs trajectory‑locked frozen‑tower optimization to reduce computational cost, and adapts objectives to each modality’s dynamics. The accompanying 5DAV dataset provides difficulty‑controllable, decoupled training samples, and experiments on JavisBench and VABench show AV‑GRPO surpasses LTX‑2.3 in generation quality, semantic alignment, and synchronization.
AVC-Reward introduces an event‑centric framework for evaluating audio‑video consistency in joint generation models. It separates audio and visual events, assesses them independently, cross‑validates cross‑modal relations, and aggregates results into a three‑level metric covering global narrative harmony, event‑level synchronization, and fine‑grained attribute alignment. The resulting AVC‑Reward model aligns closely with human judgments, offers interpretable failure localization, and enhances consistency when used as a reinforcement‑learning signal.
AVC-Reward introduces an event‑centric framework for evaluating audio‑video consistency, breaking down scenes into individual audio and visual events and assessing them at three levels: global narrative harmony, event‑level temporal synchronization, and fine‑grained attribute alignment. The metric, instantiated as a trainable reward model, is validated against human judgments using the AVC‑Benchmark, which contains diverse positive and negative samples from various generation models. AVC‑Reward outperforms existing automatic methods, offers interpretable failure localization, and enhances consistency when used as a reinforcement‑learning signal.
arXiv:2606. 01703v1 Announce Type: cross Abstract: We address the challenge of generating high-fidelity, long-form soundtracks that remain coherent across scene transitions.
arXiv:2608. 18607v2 Announce Type: replace Abstract: Using reinforcement learning to post-train joint video-audio generation models requires a reward signal.
arXiv:2603. 00610v3 Announce Type: replace-cross Abstract: While music generation models have evolved to handle complex multimodal inputs mixing text, lyrics, and reference audio, evaluation mechanisms have lagged behind.
arXiv:2606. 30642v1 Announce Type: cross Abstract: Full-length song generation must preserve coherence and musicality, render detailed vocal and accompaniment acoustics, and follow lyrics and prompts.
PRISM‑Bench is an audio‑centric diagnostic benchmark for text‑to‑audio‑video generation, built from 900 human‑verified samples. It evaluates audio along two axes—audio type (speech, music, sound) and sound‑source visibility (on‑screen vs. off‑screen)—across four perceptual dimensions (audio‑visual coherence, audio quality, audio expressiveness, and prompt following) using 35 fine‑grained criteria. The benchmark employs an enhanced MLLM‑as‑a‑Judge protocol that aligns strongly with human raters, revealing a performance gap between frontier and open‑source T2AV models and highlighting overfitting to perceptual fidelity while struggling with complex grounding and control tasks, especially for music and synchronized on‑screen audio.
StreamAV-Bench is the first comprehensive benchmark designed for streaming audio‑video generation, addressing the limitations of existing benchmarks that focus on completed sequences. It introduces a unified evaluation framework with a progressive track for instruction adherence and long‑horizon stability, and an interactive track for responsive interaction and state retention. The benchmark includes 32 fine‑grained, expert‑verified evaluation cases and evaluates 13 representative systems, revealing temporal drift in progressive generation and responsiveness bottlenecks in interactive control.
arXiv:2604.15086v3 Announce Type: replace-cross Abstract: Recent advances in video-to-audio (V2A) generation enable high-quality audio synthesis from visual content, yet achieving robust and fine-gra...
arXiv:2609.37200v1 Announce Type: new Abstract: Multi-reward guided reinforcement learning (i.e., RL) offers a promising way to improve joint audio-video diffusion models along several complementary...