arXiv AI
Video-to-Music Generation for Gameplay Videos
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 18
Distinguishing AI-Generated Music from Edited Audio as a Hard-Negative Robustness Task
arXiv:2608. 14916v1 Announce Type: cross Abstract: AI-generated music detectors are commonly evaluated against original songs, but real-world uploads are often remixed, re-encoded, pitch-shifted, or otherwise edited.
arXiv AI
Sep 7
PRISM-Bench: An Audio-Centric Diagnostic Benchmark for Text-to-Audio-Video Generation
PRISM‑Bench is an audio‑centric diagnostic benchmark for text‑to‑audio‑video generation, built from 900 human‑verified samples. It evaluates audio along two axes—audio type (speech, music, sound) and sound‑source visibility (on‑screen vs. off‑screen)—across four perceptual dimensions (audio‑visual coherence, audio quality, audio expressiveness, and prompt following) using 35 fine‑grained criteria. The benchmark employs an enhanced MLLM‑as‑a‑Judge protocol that aligns strongly with human raters, revealing a performance gap between frontier and open‑source T2AV models and highlighting overfitting to perceptual fidelity while struggling with complex grounding and control tasks, especially for music and synchronized on‑screen audio.
arXiv AI
Aug 17
Content Based Video Narration of Gameplay with Vision Language Models
arXiv:2608. 14016v1 Announce Type: cross Abstract: Live game commentary is scarce: it exists for professional esports broadcasts and almost nowhere else.
arXiv Machine Learning
Aug 17
SALSA-V: Shortcut-Augmented Long-form Synchronized Audio from Videos
arXiv:2510. 02916v2 Announce Type: replace-cross Abstract: We propose SALSA-V, a multimodal video-to-audio generation model capable of synthesizing highly synchronized, high-fidelity long-form audio from silent video content.
arXiv Computer Vision
Aug 28
StreamAV-Bench: A Comprehensive Benchmark for Streaming Audio-Video Generation
StreamAV-Bench is the first comprehensive benchmark designed for streaming audio‑video generation, addressing the limitations of existing benchmarks that focus on completed sequences. It introduces a unified evaluation framework with a progressive track for instruction adherence and long‑horizon stability, and an interactive track for responsive interaction and state retention. The benchmark includes 32 fine‑grained, expert‑verified evaluation cases and evaluates 13 representative systems, revealing temporal drift in progressive generation and responsiveness bottlenecks in interactive control.
arXiv AI
Sep 2
TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data
TUTTI is a new pre‑training framework for audio‑to‑score transcription that uses a large, fully synthetic multi‑instrument dataset generated by a symbolic music model. The approach trains a standard Transformer encoder‑decoder on these synthetic audio‑score pairs, producing a stronger foundational representation than single‑instrument training. When fine‑tuned on real datasets, TUTTI surpasses prior methods, achieving state‑of‑the‑art results and demonstrating strong cross‑instrument transferability.