Many applications of physical AI run within finite or closed physical worlds with a limited set of physical laws governing object behavior. Examples include robots working in a warehouse and agents mo...
arXiv:2607. 05352v1 Announce Type: cross Abstract: We introduce the first multiplayer world model for highly dynamic environments governed by complex physical interactions.
The paper introduces Kuration SDK, a toolkit for curating raw gameplay data to address the Virtual2Real gap in action‑conditioned world models. It demonstrates that conventional metrics like FVD, LPIPS, and JEDi do not reliably predict playability, using CounterStrike data to show divergent model performance despite similar scores. By providing curation strategies and diagnostic tools, the SDK helps uncover root causes of the gap and accelerate the creation of sample‑efficient training datasets.
The paper introduces CoDeR, a new paradigm for world modeling that explicitly builds an executable world using code rather than relying solely on visual observations. CoDeR translates high‑level concepts into structured world rules, executable dynamics, and perceptual observations through five complementary roles, enabling long‑term memory, open‑ended interactions, autonomous world evolution, and persistent multi‑agent dynamics. Experiments show that this framework extends the capabilities of existing world models and achieves state‑of‑the‑art performance across multiple evaluation settings.
arXiv:2511.09057v4 Announce Type: replace-cross Abstract: A world model is a cognitive simulator of the real-world environment allowing biological agents to reason about how the world evolves, whethe...
arXiv:2609.25652v1 Announce Type: new Abstract: Recent game world models support realistic visual simulation and interactive gameplay based on player inputs. However, they typically learn environment...
arXiv:2608.24855v1 Announce Type: new Abstract: Latent world models are inherently strong encoders that transform image pixel to latent embedding, yet existing world models still rely on online traje...
WorldMind is a decoupled framework for state-aware NPC behavior in game world models, separating interactive world modeling into four layers: Understanding, Decision, Control, and Generation. It constructs a compact state from generated frames, reasons over it to plan NPC actions, translates actions into temporally aligned conditions, and synthesizes visual outcomes. Experiments on the newly introduced BOSS-140K dataset show that WorldMind achieves more tactically appropriate and coherent NPC behavior than baseline models in about 70% of pairwise comparisons.
arXiv:2609.37907v1 Announce Type: new Abstract: Video games offer scalable environments for studying perception and control in embodied agents.Abundant online gameplay videos could supply demonstrati...
MineExplorer is a benchmark designed to assess the open‑world exploration abilities of multimodal large language models (MLLMs) in Minecraft. It filters out tasks that rely heavily on Minecraft‑specific knowledge, organizes tasks into ReAct‑style capabilities, and composes atomic tasks into implicit multi‑hop challenges. A multi‑agent synthesis workflow creates reliable task graphs, sandbox scenes, and rule‑based milestone evaluators, and human evaluation confirms its superiority over a single‑agent baseline. Experiments show that while advanced MLLMs can handle many single‑hop tasks, they struggle with longer trajectories that require coordinating hidden prerequisites, and larger models or different thinking modes do not consistently improve performance.
Synthesizing realistic Human-Object Interactions (HOI) is critical for creating embodied avatars and functional virtual environments. However, current data-driven approaches primarily rely on motion capture datasets, which are expensive to scale and limited in functional diversity.
PlaySuite is a large-scale benchmark that evaluates interactive visual intelligence by using over 5,000 open-source video games from platforms like PyWeek and itch.io. The benchmark covers diverse game engines (Pygame, HTML5, Godot, Unity) and introduces a unified closed-loop interaction framework and a Video-LLM-as-a-judge protocol to standardize progress measurement. Evaluation of fourteen recent models shows a perception-action gap, with strong reasoning but poor sustained progress, spatial grounding, action execution, and self-correction.