The paper introduces Gibson, a virtual environment designed to develop real‑world perception for active agents. Gibson virtualizes real spaces, containing over 1,400 floor plans from 572 buildings, and includes a synthesis mechanism called "Goggles" that allows models trained in simulation to be deployed directly in the real world without additional domain adaptation. The environment also embeds agents within realistic physics and spatial constraints to support embodied learning.
arXiv:2602. 09153v2 Announce Type: replace-cross Abstract: Simulation has become a key tool for training and evaluating home robots at scale, yet existing environments fail to capture the diversity and physical complexity of real indoor spaces.
SceneBloom is an agentic framework that generates realistic, varied, and simulation-ready indoor environments for embodied learning. It combines diffusion-based visual layout sampling with a contract-guided 3D compiler that uses extracted relations for geometric fitting and pose correction. The method outperforms baselines across five room types, achieving higher visual quality, physical stability, and better alignment with real room distributions, and improves object navigation policy success rates on real scans.
arXiv:2609.37089v1 Announce Type: new Abstract: Real-world videos provide rich demonstrations of manipulation, but turning them into reusable robot skills requires visually aligned environments, exec...
Real-to-sim conversion for robotic interaction with objects remains labor-intensive because it requires more than visual reconstruction: a streamlined real2sim process must recover scene geometries and object states, infer physical parameters, and assemble actors, objects, cameras, poses, and trajectories into a runnable physical simulation. Today this process still depends on manual tuning of visual foundation models, mesh cleanup, coordinate-frame alignment, and brittle workflow glue across visual perception tools and simulators.
arXiv:2607. 19190v1 Announce Type: cross Abstract: Real-to-sim conversion for robotic interaction with objects remains labor-intensive because it requires more than visual reconstruction: a streamlined real2sim process must recover scene geometries and object states, infer physical parameters, and assemble actors, objects, cameras, poses, and trajectories into a runnable physical simulation.
AgentGarten is a framework that links simulators and game engines with a shared neural renderer to create real‑time interactive virtual worlds. It uses a pretrained video model adapted to geometry conditions, refined with Adversarial Forcing, to produce realistic visual observations while maintaining consistent state and rules. The system enables agents to learn efficiently—achieving comparable performance in only four training rounds versus millions in traditional reinforcement learning—by distilling experience into playbooks that successive agents can inherit and improve upon.
The paper introduces RoboHarness, a simple robotic harness that equips large language models (LLMs) with a visual‑geometric control panel for direct manipulation of embodied tasks. By enabling LLM agents to understand and invoke actions without complex vision‑language‑action (VLA) training, RoboHarness allows models such as Qwen3.8‑Flash‑Next to outperform state‑of‑the‑art VLA models on the BEHAVIOR Challenge 2025 tasks. The authors provide an open‑source implementation of the system.
arXiv:2604.18484v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models drive next-generation autonomous systems, but training them requires scalable, high-quality annotations from co...
arXiv:2610.14018v1 Announce Type: cross Abstract: The rise of large language models (LLMs) has opened a path toward generalist embodied AI, raising the question of how LLMs can act in the physical wo...
“SceneSmith” system uses collaborative AI agents to create realistic 3D environments of places like kitchens, hotels, and living rooms, where robots can simulate everyday chores.
arXiv:2605. 14398v2 Announce Type: replace Abstract: World models have emerged as a powerful paradigm for building interactive simulation environments, with recent video-based approaches demonstrating impressive progress in generating visually plausible dynamics.
arXiv:2608. 09848v1 Announce Type: new Abstract: The development of embodied Intelligent Virtual Agents (IVAs) that have cognitive capabilities in real-time interactive virtual environments remains a challenge, even with today's advancements in technology.