LiteReality-Agent is an agentic system that reconstructs real indoor environments into realistic, articulated, and simulation-ready 3D scenes from RGB‑D scans. It treats reconstruction as a coding problem, where a coding agent iteratively edits a Python script (Room.py) using specialized tools, while an observe‑edit‑verify harness ensures evidence gathering, measurement, verification, layout optimisation, simulation readiness, and quality control. The system outperforms recent models like Astra and Fable in geometric accuracy, visual realism, and simulation compatibility, and its modular framework is positioned as a robust foundation for future agent‑driven reconstruction tasks.
By Zhening Huang, Yueyan Li, Johnathan Chiu, Xiaoyang Lyu, Matt Zhou, Yuxin Yao, Joan Lasenby, Shangzhe Wu
arXiv:2610.03715v1 Announce Type: cross
Abstract: We introduce 4DCodeBench, a benchmark for 4D inverse graphics through code generation, in which agents reconstruct dynamic scenes from video as execu...
By Ruihong Shen, \v{Z}iga Kova\v{c}i\v{c}, Peter Kulits, Xingrui Wang, Zizhang Li, Joshua B. Tenenbaum, Alan Yuille, Jieneng Chen, Jiajun Wu
arXiv:2609.36380v1 Announce Type: new
Abstract: A 3D scene reconstructed from a single image is most useful when represented not as a rendering or a fixed 3D output, but as an explicit scene program...
By Xirui Li, Peng Shi, Mingwen Dong, Sheng Zhang, Zhuoyan Xu, Dongkyu Lee, Shuaichen Chang, Yi Xiang, Lin Pan, Jiarong Jiang
arXiv:2610.04432v2 Announce Type: replace
Abstract: Building interactive simulators from real-world observations is a promising way to scale embodied data, but current pipelines still rely heavily on...
By Jinzhou Tang, Zijun Zhang, Jing Yang, Yuchen Yan, Kun Zhou, Lingjun Mao, Ruobing Han, Jinglin Cao, Wenpeng Xu, Lukun He, Minghao Fu, Fan Feng, Biwei Huang
arXiv:2607. 01766v1 Announce Type: new Abstract: LLM agents are increasingly used to translate natural language into 3D scenes in a procedural way, but existing systems focus on static output.
By Chunjiang Liu, Xiaoyuan Wang, Haoyu Chen, Yizhou Zhao, Ming-Hsuan Yang, L\'aszl\'o A. Jeni
arXiv:2605. 14398v3 Announce Type: replace Abstract: Video-based world models generate visually plausible rollouts, but since they infer dynamics in latent states, they enforce no explicit physical constraints: contacts drift, shapes distort, and motion loses consistency.
By Hongyu Wang, Jingquan Wang, Ashvin Anilkumar, Bocheng Zou, Radu Serban, Dan Negrut
LabWorld is a Real2Sim framework that reconstructs interactive 4D laboratory worlds from monocular RGB video, producing reusable 3D assets, metric scenes, time‑varying object states, and procedure events. By leveraging multi‑frame observations and motion reconstruction that combines visual, geometric, and event constraints, LabWorld outperforms existing baselines on real pipetting and vortexing tasks, achieving higher Mask IoU and PCK‑AUC scores. The framework enables explicit, editable state representation for replay, inspection, and future embodied scientific systems.
By Suozhi Huang, Kelsey Fu, Binxu Li, Luna Lyu, Kaize Wu, Wenbo Lu, David Smerkous, Jiahan Li, Siqi Zhu, Zhizhou Sha, Jiacheng Guo, Zhixuan Liang, Shilong Liu, Le Cong, Mengdi Wang
SceneTeract is a verification interface that separates semantic action understanding from physical feasibility in indoor 3D scenes. It decomposes activities into atomic actions and performs explicit geometric checks to determine executability, providing diagnostic traces for failures. The system reveals widespread functional and accessibility issues in synthetic scenes, shows that existing VLMs over‑predict action feasibility, and improves VLM performance through post‑training with verifier feedback, with benefits that generalize to real‑world scenes.
By L\'eopold Maillard, Francis Engelmann, Tom Durand, Boxiao Pan, Yang You, Leonidas Guibas, Maks Ovsjanikov
SceneAgent is an agentic pipeline that transforms 3D and 4D visual captures—such as 3D Gaussian Splatting, photogrammetry, LiDAR, AI‑generated scenes, and dynamic 4D videos—into interactive simulation environments. It coordinates object reconstruction, physical modeling, part decomposition, articulation, and deformable simulation, and then generates diverse object variants and scripted demonstrations to support policy learning. In benchmark tests, SceneAgent outperforms SimFoundry on cluttered bin scenes and dramatically improves dynamic IoU on 4DCodeBench, while also enabling simulated interactions and preliminary real‑robot policy transfer.
By Luke Hollis, Tianxing Fan, Heng Yang
LLM agents are increasingly used to translate natural language into 3D scenes in a procedural way, but existing systems focus on static output. Dynamic 4D scenes from text alone, in which liquids flow, particles emit, rigid bodies cascade, and articulated mechanisms move, remain largely unexplored despite their value as editable content and as physics-grounded training data for video generation and embodied AI.
arXiv:2610.08791v1 Announce Type: new
Abstract: Video world models can produce visually convincing yet physically inconsistent sequences, raising concerns about their reliability for prediction and p...
By Mingju Gao, Qingle Liu, Yuzhao Peng, Xinjie Lin, Ziming Qin, Zheng Jiang, Wenyi Li, Calvin Xiao, Youjie Zheng, Kaisen Yang, Qinhuai Na
Code4Scene is a benchmark that evaluates coding agents on constructing and editing 3D scenes in Unreal Engine. It tests agents on two tasks: construction, where they must build a scene from open‑ended language, and editing, where they must recover a target scene from reference images while preserving everything else. The benchmark measures task fulfillment, artifact integrity, and physical validity, revealing that construction and editing performance are correlated but not interchangeable, with agents struggling most with spatial composition and precise edits.
By Xiaokang Ye, Siddhant Hitesh Mantri, Zimeng Chen, Edward Zhang, Zhaoxu Zheng, Yuanheng Li, Yizhao Chen, Tianyang Huang, Lianhui Qin