arXiv Machine Learning
PlayGen2: Learning Real-Time Playable Worlds with Explicit 3D State
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computer Vision
Sep 10
Programmable World Model
arXiv:2609.10540v1 Announce Type: new Abstract: Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent w...
arXiv Computer Vision
Sep 1
Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory
arXiv:2608.29910v1 Announce Type: new Abstract: Interactive world models extend video generation from offline clip synthesis toward persistent simulation of interactive virtual worlds, enabling appli...
arXiv Computer Vision
Oct 28
BoxWorld: Let World Engines Control and Video Generators Render
BoxWorld introduces a novel approach to real‑time interactive video worlds by decoupling interaction reasoning from generative rendering. A coding agent builds a Box World Engine that applies user‑defined rules to determine interaction outcomes and exposes object states via a lightweight interface of layout boxes, persistent attributes, and dynamic status variables. A separate Box‑State Generative Renderer then autoregressively renders these evolving states into video, enabling online control over object motion, state switches, and conditional events. The system employs a three‑stage training strategy—learning object grounding from open‑domain videos, state transitions from gameplay videos, and rollout distillation on self‑generated histories—to handle real‑time state updates while maintaining visual coherence. BoxWorld outperforms end‑to‑end world models and remains competitive across multiple public benchmarks, with a new PairedStateBench benchmark for fine‑grained interaction control.
arXiv AI
Jun 4
Beyond Pixel Histories: World Models with Persistent 3D State
arXiv:2603. 03482v2 Announce Type: replace-cross Abstract: Interactive world models continually generate video by responding to a user's actions, enabling open-ended generation capabilities.
arXiv Machine Learning
Oct 15
StateCraft: Augmenting Video World Models with Abstract-State Reasoning
StateCraft introduces a new architecture that separates a persistent, updatable abstract state from the visual state in video world models. By pairing a language model with a video diffusion model, it tracks entity changes over time and conditions visual generation on this abstract state. The approach is evaluated on generative interfaces and a new benchmark, showing improved generation quality, memory usage, and controllability, while enabling direct editing of unseen states and extending to spatial 3D environments.
arXiv AI
Sep 11
Valerant: An Automatic Navigable Game Map Generator via Action-Conditioned World Model Exploration
Valerant is a training‑free framework that turns a pretrained action‑conditioned world model into a world action model (WAM) capable of exploring and building 3D game maps. By combining predictive visual rollouts with SLAM‑based spatial reconstruction and exploration‑driven action selection, it converts a single image into a persistent, navigable 3D geometry. This approach extends WAM‑based interaction beyond 2D visual simulation and reduces manual effort in 3D game‑map creation.