arXiv Computer Vision

Frontier3DBench: Benchmarking Coding Agents for 3D Creation across Objects, Scenes, and Interactive Open Worlds

Frontier3DBench is a unified benchmark for coding agents that generate, reconstruct, and edit 3D content, comprising 145 tasks across nine categories covering objects, scenes, and interactive open worlds. The benchmark evaluates appearance, structure, and behavior, revealing that GPT‑6.1‑sol excels at object and scene construction while GPT‑6‑astra leads interactive world building. Results show that agents perform better at text‑driven generation than at precise reference reconstruction, and that coordinated activity completion remains a significant challenge.

arXiv AI
Oct 16

3DCodeVerse: Exploring 3D Worlds as Code

The paper introduces 3DCodeVerse, a research ecosystem that allows coding agents to generate editable and verifiable 3D content at scale. It comprises a large curated dataset of 3D executable code (3DCodeVerse‑2M), a multi‑agent harness that automatically evaluates and refines generated programs using execution checks and spatial measurements, and empirical studies that establish baselines and practical guidance for 3D coding. Experiments show that fine‑tuning on the dataset improves open‑source models’ 3D coding abilities, and the harness outperforms Claude Design in pairwise human evaluations when used with Claude Opus 5.5.

By Yipeng Gao, Ziyao Zeng, Jinfa Huang, Yijiang Li, Haolin Xiong, Wang Qin, Xinlei Yu, Youheng Yao, Jie Zhang, Kun-Yu Lin, Rong Liu, Yinyuan Zhao, Yajie Zhao, Meiqi Guo, Jiebo Luo, Lei Shu, Yunhao Ge, Xinyu Hu, Laurent Itti
arXiv AI
Sep 30

Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes

Code4Scene is a benchmark that evaluates coding agents on constructing and editing 3D scenes in Unreal Engine. It tests agents on two tasks: construction, where they must build a scene from open‑ended language, and editing, where they must recover a target scene from reference images while preserving everything else. The benchmark measures task fulfillment, artifact integrity, and physical validity, revealing that construction and editing performance are correlated but not interchangeable, with agents struggling most with spatial composition and precise edits.

By Xiaokang Ye, Siddhant Hitesh Mantri, Zimeng Chen, Edward Zhang, Zhaoxu Zheng, Yuanheng Li, Yizhao Chen, Tianyang Huang, Lianhui Qin
arXiv Computer Vision
Oct 28

Learning to Code the World: Harness Evolution for Compositional 3D Reconstruction

The paper introduces SceneEvolve, a framework that represents 3D scenes as executable code and improves reconstruction by evolving a harness around a small open‑source agent. SceneEvolve combines a compositional semantic action interface (SceneAct) with a harness evolution process (SceneHarness‑RSI) that uses a stronger teacher agent for iterative refinement. The resulting harness allows the small Qwen3.8‑27B model to reconstruct unseen scenes without parameter updates, achieving higher reconstruction quality and lower inference time than larger proprietary models across multiple datasets.

By Haoyu Yao, Chenguo Lin, Ruize Du, Yadong Mu
arXiv Computer Vision
Oct 8

Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos

arXiv:2610.04432v2 Announce Type: replace Abstract: Building interactive simulators from real-world observations is a promising way to scale embodied data, but current pipelines still rely heavily on...

By Jinzhou Tang, Zijun Zhang, Jing Yang, Yuchen Yan, Kun Zhou, Lingjun Mao, Ruobing Han, Jinglin Cao, Wenpeng Xu, Lukun He, Minghao Fu, Fan Feng, Biwei Huang
arXiv Computer Vision
Nov 6

WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes

WorldAct is a framework that transforms static, monolithic 3D worlds generated by systems like Marble into editable, interaction‑ready scenes. It employs a multimodal agent to decompose the scene, identify actionable objects, reconstruct object‑level meshes, and inpaint the remaining background. The resulting scenes allow object‑level editing, collision‑aware manipulation, and embodied task execution while maintaining global coherence.

By Jichen Hu, Jiawei Guo, Jiazhong Cen, Chen Yang, Sikuang Li, Wei Shen