The paper introduces two minimal simulation foundations—SD-AgentFoundry-2D and SD-AgentFoundry-3D—for educational and rapid prototyping use with large language models (LLMs) and vision-language models (VLMs). SD-AgentFoundry-2D offers a 2D multi‑agent environment where LLM agents move, communicate, and react to local events such as fire, while SD-AgentFoundry-3D provides a 3D digital‑twin setting where a VLM interprets first‑person images to generate natural‑language movement instructions. Both frameworks run locally on macOS, Windows, and Linux, are intentionally lightweight, and are open for modification rather than being finished applications.
Large language models (LLMs) and vision-language models (VLMs) are expanding the range of behaviors that can be represented in agent-based simulations, but many contemporary platforms are difficult to...
arXiv:2607. 00989v1 Announce Type: cross Abstract: Semantic trajectory analysis has recently emerged as an approach for modeling human movement by capturing implicit patterns and behaviors through semantic information (e.
arXiv:2607. 21522v1 Announce Type: cross Abstract: Creating dynamic and physically realistic 4D worlds from natural language descriptions is both fascinating and challenging.
arXiv:2507. 09788v3 Announce Type: replace-cross Abstract: Recent advances in Large Language Models (LLM) have led to a new class of autonomous agents, renewing and expanding interest in the area.
arXiv:2605. 14398v2 Announce Type: replace Abstract: World models have emerged as a powerful paradigm for building interactive simulation environments, with recent video-based approaches demonstrating impressive progress in generating visually plausible dynamics.
LLM agents are increasingly used to translate natural language into 3D scenes in a procedural way, but existing systems focus on static output. Dynamic 4D scenes from text alone, in which liquids flow, particles emit, rigid bodies cascade, and articulated mechanisms move, remain largely unexplored despite their value as editable content and as physics-grounded training data for video generation and embodied AI.
The paper introduces CoDeR, a new paradigm for world modeling that explicitly builds an executable world using code rather than relying solely on visual observations. CoDeR translates high‑level concepts into structured world rules, executable dynamics, and perceptual observations through five complementary roles, enabling long‑term memory, open‑ended interactions, autonomous world evolution, and persistent multi‑agent dynamics. Experiments show that this framework extends the capabilities of existing world models and achieves state‑of‑the‑art performance across multiple evaluation settings.
arXiv:2607. 01766v1 Announce Type: new Abstract: LLM agents are increasingly used to translate natural language into 3D scenes in a procedural way, but existing systems focus on static output.
4DSynth is a controllable procedural system that transforms natural-language descriptions, blueprint masks, or single photographs into editable 4D environments featuring explicit geometry, animated actors, collision-free trajectories, and physics-ready simulation states. The system unifies animation, camera planning, rendering, and task generation within a single geometry-grounded representation, enabling scalable creation of dynamic embodied simulation scenes. Using 4DSynth, the authors built 4DSynth-Nav, an interactive navigation benchmark that demonstrates the reproducibility and tunability of procedural failures across vision‑language models.
arXiv:2602. 09153v2 Announce Type: replace-cross Abstract: Simulation has become a key tool for training and evaluating home robots at scale, yet existing environments fail to capture the diversity and physical complexity of real indoor spaces.
4DSynth is a controllable procedural system that transforms natural-language descriptions, blueprint masks, or single photographs into editable 4D environments featuring explicit geometry, animated actors, collision-free trajectories, and physics-ready simulation states. The system unifies animation, camera planning, rendering, and task generation within a single geometry-grounded representation, enabling scalable creation of diverse, interactive scenes. Using 4DSynth, the authors built 4DSynth-Nav, an interactive navigation benchmark that demonstrates the reproducibility of failures and tunable difficulty across three tiers for vision‑language models.
arXiv:2504. 09662v4 Announce Type: replace-cross Abstract: Multi-agent large language model simulations have the potential to model complex human behaviors and interactions.