arXiv AI

Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes

Code4Scene is a benchmark that evaluates coding agents on constructing and editing 3D scenes in Unreal Engine. It tests agents on two tasks: construction, where they must build a scene from open‑ended language, and editing, where they must recover a target scene from reference images while preserving everything else. The benchmark measures task fulfillment, artifact integrity, and physical validity, revealing that construction and editing performance are correlated but not interchangeable, with agents struggling most with spatial composition and precise edits.

arXiv Computer Vision
Nov 9

Frontier3DBench: Benchmarking Coding Agents for 3D Creation across Objects, Scenes, and Interactive Open Worlds

Frontier3DBench is a unified benchmark for coding agents that generate, reconstruct, and edit 3D content, comprising 145 tasks across nine categories covering objects, scenes, and interactive open worlds. The benchmark evaluates appearance, structure, and behavior, revealing that GPT‑6.1‑sol excels at object and scene construction while GPT‑6‑astra leads interactive world building. Results show that agents perform better at text‑driven generation than at precise reference reconstruction, and that coordinated activity completion remains a significant challenge.

By Xinjie Lin, Qingle Liu, Wenyi Li, John Lam, Xiaoqiu Wang, Deyao Hong, Mingju Gao, Zheng Jiang, Yizhe Chi, Youjie Zheng, Calvin Xiao, Kaisen Yang, Huan-ang Gao
arXiv Computer Vision
Aug 28

Procedura: Agentic 3D Modeling with Procedural Control

Procedura is a new 3D modeling agent that treats 3D shape as code, using a large language model to generate a procedural assembly from a text prompt. It constructs an assembly graph, writes a parametric program with named parts and typed mates, and verifies each part through compile, mate, and connectivity checks before adding it. A vision critic refines the assembly step‑by‑step, and the resulting program includes per‑part materials and simulator‑validated articulation, producing sharp edges and editable, part‑structured outputs that outperform existing native 3D generators on P3D‑Bench and MechBench‑36.

By Youtian Lin, Yikang Yang, Zhanpeng Hu, Mengqi Zhou, Feihu Zhang, Xun Cao, Jiaheng Liu, Yao Yao
arXiv AI
Aug 21

ChronoAgentic: A Code-based Multi-Agent World Simulator for Physically Grounded Simulation Construction

arXiv:2605. 14398v3 Announce Type: replace Abstract: Video-based world models generate visually plausible rollouts, but since they infer dynamics in latent states, they enforce no explicit physical constraints: contacts drift, shapes distort, and motion loses consistency.

By Hongyu Wang, Jingquan Wang, Ashvin Anilkumar, Bocheng Zou, Radu Serban, Dan Negrut
arXiv AI
Oct 16

3DCodeVerse: Exploring 3D Worlds as Code

The paper introduces 3DCodeVerse, a research ecosystem that allows coding agents to generate editable and verifiable 3D content at scale. It comprises a large curated dataset of 3D executable code (3DCodeVerse‑2M), a multi‑agent harness that automatically evaluates and refines generated programs using execution checks and spatial measurements, and empirical studies that establish baselines and practical guidance for 3D coding. Experiments show that fine‑tuning on the dataset improves open‑source models’ 3D coding abilities, and the harness outperforms Claude Design in pairwise human evaluations when used with Claude Opus 5.5.

By Yipeng Gao, Ziyao Zeng, Jinfa Huang, Yijiang Li, Haolin Xiong, Wang Qin, Xinlei Yu, Youheng Yao, Jie Zhang, Kun-Yu Lin, Rong Liu, Yinyuan Zhao, Yajie Zhao, Meiqi Guo, Jiebo Luo, Lei Shu, Yunhao Ge, Xinyu Hu, Laurent Itti
arXiv Computer Vision
Oct 16

DeSpatial: The devil of Agentic Spatial Reasoning Lies in 3D Itself

The paper introduces DeSpatial, a framework that separates spatial understanding into perception and reasoning. It uses a shared 3D perception tool with joint segmentation–depth co-refinement and cross-view instance fusion to produce a unified, metric, gravity-aligned 3D scene. DeSpatial achieves state‑of‑the‑art results on VSI‑Bench, MindCube‑Tiny, ViewSpatial, and VSI‑SUPER, demonstrating that the main bottleneck in spatial reasoning is precise 3D perception.

By Yifei Liu, Xiaolong Zhou, Ziyang Gong, Yifan Zhou, Yuanyuan Gao, Yuning Gong, Yuchen Yang, Hongjie Zhang, Xue Yang, Xiao Sun, Zhihang Zhong
arXiv AI
3d ago

VibePrevisBench: Evaluating Agentic Coding for 4D Scene Generation in Blender

VibePrevisBench is a new benchmark for evaluating coding agents that reconstruct 4D scenes in Blender from brief inputs and render proxy videos. It contains 242 tasks across four input types—scripts, photos, continuations, and reference videos—and assesses each output using a rule‑based scene check and a VLM‑based screen check. Analysis of 11 leading agents shows that while they excel at basic camera work, they struggle with multimodal spatial understanding, challenging requests, and self‑correction of errors.

By Yuzhe Chen, Muxi Diao, Lele Yang, Wenbin Xu, Xuyang Liu, Qile Su, Changyi Liu, Jiankang Chen, Kaiyu Jiang, Haoyu Zhang, Hongyi Fu, Jianxiong Wang, Tianke Zhang, Haonan Fan, Yingxin Li, Bingyi Li, Xu Wang, Xuanyu Zheng, Bin Wen, Tingting Gao, Wentao Zhang, Jiale Cao
arXiv AI
Sep 25

PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slides

PPTBench is a new benchmark that tests coding agents’ ability to reconstruct scientific flow diagrams from arXiv papers into editable PowerPoint slides. The dataset contains 500 tasks, each requiring agents to produce a single PPTX page with native, editable objects, and a four‑stage Agentic Judge evaluates validity, semantic correctness, rendering quality, and fine‑grained visual quality. Across 31 model configurations, the best score is 67.80, with a median of 19.47, showing that while agents can generate valid PPTX files, they still struggle with semantic and visual accuracy, especially text details.

By Xiaoqiu Wang, Yizhe Chi, Wenyi Li, Deyao Hong, Zhihan Shan, Mingju Gao, Kaisen Yang, Youjie Zheng, Calvin Xiao, Qinhuai Na
arXiv Computer Vision
Nov 2

DyCraft: An Agentic System for Long-Horizon 4D Scene Generation

DyCraft is an agentic system that generates long‑horizon 4D animated 3D scenes by decomposing the task into a detailed plan, a static scene, and a 3D storyboard of key poses. Each storyboard frame is a static scene that the agent can build and debug independently, allowing local motion to be authored with verified start and end states. In evaluations on 24 multi‑shot animation tasks, DyCraft reduces physical violations and produces higher‑quality scenes that are more often preferred by human raters compared to native GPT‑6 agents and prior baselines.

By Jeff Tan, Hang Dai, Zeqian Long, Yanzhe Lyu, R. Kenny Jones, Shangzhe Wu, Chen Geng, Jiajun Wu
arXiv AI
Oct 12

VETR: Execution-Grounded Self-Improving Agent for Symbolic Graphics Reasoning

VETR (Verified Execution Trajectories for Refinement) is a framework that enables visual agents to recursively improve symbolic graphics reasoning by extracting actionable evidence from execution traces. It retains aligned program/render records and uses an execution refiner to compile observed writes and state changes into constrained alternative programs, which are then validated against captured runtime state. Evaluations on BlenderGym and BlenderBench with GPT‑5.6‑luna and Qwen3.8‑27B demonstrate that VETR consistently outperforms existing refinement strategies such as BlenderAlchemy and VIGA.

By Yingbo Ma, Kuangshi Ai, Gordon Dai, Chenglong Yu, Chaoli Wang, Hua Shen
arXiv AI
Aug 11

VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System

arXiv:2607. 27380v2 Announce Type: replace-cross Abstract: Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt.

By Haodong Li, Tianfei Ren, Xiaoxiao Ma, Chunmei Qing, Zhen Fang, Sipeng He, Ziyu Guo, Haoyu Wu, Juanxi Tian, Yihang Zou, Ruichuan An, Dongzhi Jiang, Boxue Yang, Ji Xie, Xu Huang, Wenhao Yan, Jialv Zou, Zhengrong Yue, Yaxin Luo, Xiaotong Li, Yuzhu Wang, Junyan Ye, Jinjing Zhao, Zehui Chen, Lin Chen, Renye Yan, Feng Zhao, Pheng-Ann Heng