Mallea is a system that builds, steers, and memorizes worlds from text by using a coding agent that infers object interaction requirements and plans object structure, properties, and rules for compact representations. The world evolves under these rules, with optional user actions steering it and text requests editing properties or extending capabilities while preserving unaffected state. Object texture memory retains reusable appearance for whitebox-defined objects and adapts it as those objects change, while a generative renderer combines current structure with remembered appearance to advance the world state.
arXiv:2609.28236v1 Announce Type: new Abstract: Long-horizon embodied interaction requires agents to retain and continually update information about the environment as they observe, act, and encounte...
The paper introduces CoDeR, a new paradigm for world modeling that explicitly builds an executable world using code rather than relying solely on visual observations. CoDeR translates high‑level concepts into structured world rules, executable dynamics, and perceptual observations through five complementary roles, enabling long‑term memory, open‑ended interactions, autonomous world evolution, and persistent multi‑agent dynamics. Experiments show that this framework extends the capabilities of existing world models and achieves state‑of‑the‑art performance across multiple evaluation settings.
BoxWorld introduces a novel approach to real‑time interactive video worlds by decoupling interaction reasoning from generative rendering. A coding agent builds a Box World Engine that applies user‑defined rules to determine interaction outcomes and exposes object states via a lightweight interface of layout boxes, persistent attributes, and dynamic status variables. A separate Box‑State Generative Renderer then autoregressively renders these evolving states into video, enabling online control over object motion, state switches, and conditional events. The system employs a three‑stage training strategy—learning object grounding from open‑domain videos, state transitions from gameplay videos, and rollout distillation on self‑generated histories—to handle real‑time state updates while maintaining visual coherence. BoxWorld outperforms end‑to‑end world models and remains competitive across multiple public benchmarks, with a new PairedStateBench benchmark for fine‑grained interaction control.
StateCraft introduces a new architecture that separates a persistent, updatable abstract state from the visual state in video world models. By pairing a language model with a video diffusion model, it tracks entity changes over time and conditions visual generation on this abstract state. The approach is evaluated on generative interfaces and a new benchmark, showing improved generation quality, memory usage, and controllability, while enabling direct editing of unseen states and extending to spatial 3D environments.
arXiv:2610.17928v1 Announce Type: new Abstract: Recent advances in vision-language models (VLMs) have substantially improved the capabilities of GUI agents, while memory augmentation has emerged as a...
arXiv:2609.10540v1 Announce Type: new Abstract: Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent w...
arXiv:2603. 03482v2 Announce Type: replace-cross Abstract: Interactive world models continually generate video by responding to a user's actions, enabling open-ended generation capabilities.
MemCo introduces a memory‑centric collaboration framework that equips large language model agents with both local and global memory spaces. Local memory preserves environment‑specific details, while global memory captures transferable workflows from local trajectories. During online interaction, MemCo selectively routes relevant memories based on the agent’s state and decision phase, enabling reuse of other agents’ experiences without over‑generalizing environment‑specific information.
MemoryVLN introduces a memory-driven navigation system that compresses long navigation histories and reuses past spatial experience. It uses forgetting-curve-inspired temporal sampling for long sequence compression, an ego-aware memory representation that fuses 2D semantic, 3D geometric, and 6D pose tokens, and a hierarchical memory bank for in-episode and context-level memory management. The system retrieves top‑K relevant memories to augment current visual tokens, achieving significant improvements on R2R‑CE and RxR‑CE benchmarks.
arXiv:2609.37923v1 Announce Type: new Abstract: Agents can learn from past executions, but enabling different agents to reuse and build on one another's experience remains challenging. We introduce E...
The paper introduces RoomWright, a code‑driven framework that generates 3D indoor scenes for embodied AI by focusing on functional usage rather than just visual layout. It performs usage‑driven object reasoning, treating anchors as task centers to select task‑required objects and their affordances, and compiles interactions into trigger‑condition‑effect rules that update object states. The system also addresses ambiguous object orientation through annotation‑guided usage cues, producing scenes that are executable, editable, and ready for simulation‑based policy learning.
Mem-World introduces a memory‑augmented action‑conditioned world model for robot manipulation, featuring W‑VMem—a 4D wrist‑view‑centered surfel‑indexed memory that anchors historical observations to evolving surface elements. By explicitly modeling when and where scene elements are observed, the system retrieves geometry‑aware history frames during generation, providing informative, non‑redundant context for future action predictions. Experiments demonstrate that Mem‑World produces persistent rollouts, improves policy evaluation reliability (14.5 % higher Pearson correlation with real‑world performance), and boosts long‑horizon task success rates from 58 % to 72 % using synthetic data generation.