AnyGameRenderer is a video‑to‑video generative renderer that transforms low‑quality, AI‑generated game footage—such as gray blockouts and colored primitives—into high‑quality visuals with detailed characters, motion, and effects while preserving layout, camera, and entity positions. It works across diverse game genres (tower defense, auto‑battlers, top‑down survivors, turn‑based battles, kart racing, crowd runners) and accepts plain video inputs without requiring complex engine data. Trained on ~10k paired clips from agent‑made games, Unreal Engine simulations, and real gameplay, it improves entity correspondence, raising temporal match from 50.9% to 62.9% and reducing missed count changes from 34.7% to 21.3%.
arXiv:2608.24680v1 Announce Type: new Abstract: Video games provide a scalable source of training data for video world models, offering diverse environments, complex interactions, and abundant in-the...
arXiv:2609.25652v1 Announce Type: new Abstract: Recent game world models support realistic visual simulation and interactive gameplay based on player inputs. However, they typically learn environment...
GTASA: Ground Truth Annotations for Spatiotemporal Analysis, Evaluation and Training of Video Models presents GEST-Engine, an open‑source system that deterministically renders multi‑actor video scenarios while recording exhaustive ground truth—including 3D entity and camera states, pairwise spatial relations, event‑to‑frame mappings, instance segmentation, and long descriptions—at no extra annotation cost. The authors release a 938‑video dataset, GTASA, which offers the densest spatial‑relation coverage of any video dataset, with an entity‑pair relation graph at every frame that is roughly 84× denser than current state‑of‑the‑art collections. They demonstrate the dataset’s value by showing that pretraining on GTASA improves VideoLLaMA3 captioning, and by using it to probe frozen video encoders on 11 spatio‑temporal tasks, revealing that relational understanding remains near chance. whyItMatters":"The dataset provides a uniquely dense, exact 3D ground truth for spatio‑temporal video analysis, enabling measurable progress in training and evaluating video models on relational reasoning tasks."
arXiv:2610.27999v1 Announce Type: new Abstract: Moving from controllable videos to genuinely playable worlds requires persistent world state, executable game rules, and low-latency interaction. We ou...
BoxWorld introduces a novel approach to real‑time interactive video worlds by decoupling interaction reasoning from generative rendering. A coding agent builds a Box World Engine that applies user‑defined rules to determine interaction outcomes and exposes object states via a lightweight interface of layout boxes, persistent attributes, and dynamic status variables. A separate Box‑State Generative Renderer then autoregressively renders these evolving states into video, enabling online control over object motion, state switches, and conditional events. The system employs a three‑stage training strategy—learning object grounding from open‑domain videos, state transitions from gameplay videos, and rollout distillation on self‑generated histories—to handle real‑time state updates while maintaining visual coherence. BoxWorld outperforms end‑to‑end world models and remains competitive across multiple public benchmarks, with a new PairedStateBench benchmark for fine‑grained interaction control.
arXiv:2610.28630v1 Announce Type: new Abstract: Game engines and coding agents can specify the layout, motion, and camera of a scene, yet turning such coarse structure into realistic video requires a...
VibeVideoBench is a benchmark that evaluates multimodal agents capable of generating videos by writing code to render each frame and sound. The benchmark contains 239 self‑contained tasks in English and Chinese across 12 categories, requiring agents to produce a finished MP4 with sound within one hour without using any pre‑trained image or video generation model. Evaluation is hierarchical, using program checks, 13k task‑specific rubric checks, and five quality dimensions scored by three VLM judge panels, and it shows that GPT‑6 Astra and Claude Opus 5.5 perform best among nine frontier agents.
arXiv:2610.02205v1 Announce Type: new Abstract: Programmable world models separate executable dynamics from visual generation, offering a promising foundation for next-generation game engines. Howeve...
The paper introduces a large‑scale synthetic data pipeline built on Unreal Engine to generate action‑conditioned, multi‑view video for training world models. The system operates in two stages: real‑time physics simulation records trajectories, then offline rendering produces high‑quality video. It includes a distributed production framework with task partitioning, automated filtering, and a 25‑server cluster, yielding over 2,600 hours of 1080p and 6,000 hours of 720p video from 429 levels and 40 characters.
arXiv:2610.35864v1 Announce Type: new Abstract: Diffusion video models produce striking clips, but they cannot reliably render an exact formula, a correct chart, a ruleconsistent game or narration sy...
arXiv:2607. 27380v2 Announce Type: replace-cross Abstract: Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt.
arXiv:2609.10540v1 Announce Type: new Abstract: Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent w...