AVC-Reward introduces an event‑centric framework for evaluating audio‑video consistency in joint generation models. It separates audio and visual events, assesses them independently, cross‑validates cross‑modal relations, and aggregates results into a three‑level metric covering global narrative harmony, event‑level synchronization, and fine‑grained attribute alignment. The resulting AVC‑Reward model aligns closely with human judgments, offers interpretable failure localization, and enhances consistency when used as a reinforcement‑learning signal.
arXiv:2608. 18607v2 Announce Type: replace Abstract: Using reinforcement learning to post-train joint video-audio generation models requires a reward signal.
By Yinming Huang, Shuyuan Tu, Xi Yan, Zihan Yang, Jianhua Han, Xu Hang, Yu-Gang Jiang, Zuxuan Wu
AV‑GRPO introduces a modality‑anchored diffusion reinforcement learning framework for joint audio‑video generation, addressing limitations in fidelity, text‑modality alignment, and cross‑modal synchronization. It decouples learning signals through modality‑anchored rollouts, employs trajectory‑locked frozen‑tower optimization to reduce computational cost, and adapts objectives to each modality’s dynamics. The accompanying 5DAV dataset provides difficulty‑controllable, decoupled training samples, and experiments on JavisBench and VABench show AV‑GRPO surpasses LTX‑2.3 in generation quality, semantic alignment, and synchronization.
By Zhiyu Xu, Weilong Yan, Yufei Shi, Shiyang Li, Yihao Liu, Kin-Man Lam, Yuewen Cao
arXiv:2605. 07061v2 Announce Type: replace-cross Abstract: Joint audio-video generation models are rapidly approaching professional production quality, raising a central question: do they understand audio-visual physics, or merely generate plausible sounds and frames that violate real-world consistency?
By Zijun Cui, Xiulong Liu, Hao Fang, Mingwei Xu, Jiageng Liu, Zexin Xu, Weiguo Pian, Shijian Deng, Feiyu Du, Chenming Ge, Yapeng Tian
TiTok is an audio‑visual large language model designed for multi‑segment temporal grounding in untrimmed videos. It introduces Time Token Interleaving (TTI) to align temporal perception with prediction, and uses decoupled reinforcement‑learning rewards (global, local, count, precision, format) optimized via GDPO. Evaluated on a new UnAV‑100 protocol with the CountF1 metric, TiTok achieves state‑of‑the‑art results (65.7 mIoU, 0.58 CountF1).
By Eunji Shin, Dahyun Choi, Seungyeon Jo, Yejin Hong, Jiyoung Lee
arXiv:2605. 00873v2 Announce Type: replace-cross Abstract: The rapid advancement of photorealistic Text-to-Video (T2V) generation brings in an urgent need for up-to-date evaluation methods.
By Advait Tilak, Jiwon Choi, Nazifa Mouli, Wei Le