PRISM‑Bench is an audio‑centric diagnostic benchmark for text‑to‑audio‑video generation, built from 900 human‑verified samples. It evaluates audio along two axes—audio type (speech, music, sound) and sound‑source visibility (on‑screen vs. off‑screen)—across four perceptual dimensions (audio‑visual coherence, audio quality, audio expressiveness, and prompt following) using 35 fine‑grained criteria. The benchmark employs an enhanced MLLM‑as‑a‑Judge protocol that aligns strongly with human raters, revealing a performance gap between frontier and open‑source T2AV models and highlighting overfitting to perceptual fidelity while struggling with complex grounding and control tasks, especially for music and synchronized on‑screen audio.
By Yuchen Sun, Qian Yang, Jun Wang, Detai Xin, Guoqiao Yu, Guanglu Wan, Qi Jia
arXiv:2605. 07061v2 Announce Type: replace-cross Abstract: Joint audio-video generation models are rapidly approaching professional production quality, raising a central question: do they understand audio-visual physics, or merely generate plausible sounds and frames that violate real-world consistency?
By Zijun Cui, Xiulong Liu, Hao Fang, Mingwei Xu, Jiageng Liu, Zexin Xu, Weiguo Pian, Shijian Deng, Feiyu Du, Chenming Ge, Yapeng Tian
The paper introduces SCR-V2A, a video-to-audio generation framework that focuses on Semantic-Critical Regions (SCRs)—localized visual cues directly related to sound events. SCR-V2A first identifies these regions via sound-related visual localization, then encodes them with an adapter-based dual visual encoder to produce a control representation that captures both global context and fine-grained sound evidence. This representation conditions an audio generation model, and experiments on in-domain and out-of-domain benchmarks show improved distribution matching and audio quality with few trainable parameters.
By Yun Fang, Wei Guo, Zhuonan Liang, Heng Wang, Weidong Cai
arXiv:2609.15562v1 Announce Type: cross
Abstract: As generative models continue to advance, AI-generated content (AIGC) is becoming increasingly realistic, weakening the artifact cues commonly exploi...
By Bo Zheng, Kangran Zhao, Xiaoyu Zhang, Weinan Guan, Zhiheng Li, Yize Chen, Haizhou Li, Qingshan Liu, Siwei Lyu, Baoyuan Wu
arXiv:2605. 00873v2 Announce Type: replace-cross Abstract: The rapid advancement of photorealistic Text-to-Video (T2V) generation brings in an urgent need for up-to-date evaluation methods.
By Advait Tilak, Jiwon Choi, Nazifa Mouli, Wei Le
arXiv:2608. 04479v1 Announce Type: cross Abstract: Text-to-audio (TTA) generation has recently achieved remarkable progress in synthesizing realistic audio from natural language descriptions.
By Jinting Wang, Yuguang Yang, Shengyu Li, Yan Rong, Shan Yang, Xiaoda Yang, Li Liu
arXiv:2607. 09973v1 Announce Type: cross Abstract: Industrial sound design requires audio generation systems that not only produce realistic audio, but also preserve the perceptual identity of a reference, support controllable variation, and remain efficient for practical workflows.
By M\'elodie Desbos, Yara Bahram, Eric Granger, Mohammadhadi Shateri
SsgCaps is a publicly available dataset of human-engineered sound scenes, each paired with a precisely structured prompt that guides the sampling process. The prompts are drawn from a predefined action-based typology, enabling extensive yet plausible sampling. A comparative quantitative analysis shows only small differences between the open and private versions, supporting the recommendation of the open version for benchmarking sound scene generation algorithms.
By Modan Tailleur (LS2N), Junwon Lee (LS2N), Laurie M Heller (LS2N), Mathieu Lagrange (LS2N), Keunwoo Choi, Brian McFee, Keisuke Imoto, Yuki Okamoto
arXiv:2609.38444v1 Announce Type: new
Abstract: Text- and image-conditioned world generators can create visually rich 3D environments, yet these worlds often remain silent or rely on soundtracks synt...
By Duowen Chen, Jinjin He, Gouthaman KV, Sandeep Bangalore Venkatesh, Bo Zhu
arXiv:2609.40317v1 Announce Type: new
Abstract: While talking head generation has advanced rapidly, generating natural listener behavior in dyadic conversations, which know when to react, how to reac...
By Zikai Liao, Yumin Suh, Yi Ouyang, Yi-Lun Lee, Yi-Hsuan Tsai, Zhaozheng Yin
arXiv:2610.19290v1 Announce Type: new
Abstract: We present Kandinsky S2V 1.0, an open family of speech-to-video models for controllable audio-driven human animation, comprising Kandinsky S2V 1.0 Lite...
By Nikolay Semenov, Artem Riabinin, Sergey Bassenko, Anna Averchenkova, Julia Agafonova, Anastasia Aliaskina, Kirill Kozlov, Lev Novitskiy, Nikita Samsonov, Nikolai Gerasimenko, Denis Dimitrov
arXiv:2605.07593v2 Announce Type: replace
Abstract: Real-world audio-visual understanding requires chaining evidence that is sparse, temporally dispersed, and split across the visual and auditory str...
By Hengyi Feng, Hao Liang, Mingrui Chen, Bohan Zeng, Meiyi Qiang, Zhengyang Zhao, Zimo Meng, Zeang Sheng, Wentao Zhang