LEGO-Anything: Coding Agents for 3D Scene Reconstruction
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Frontier3DBench is a unified benchmark for coding agents that generate, reconstruct, and edit 3D content, comprising 145 tasks across nine categories covering objects, scenes, and interactive open worlds. The benchmark evaluates appearance, structure, and behavior, revealing that GPT‑6.1‑sol excels at object and scene construction while GPT‑6‑astra leads interactive world building. Results show that agents perform better at text‑driven generation than at precise reference reconstruction, and that coordinated activity completion remains a significant challenge.
The paper introduces SceneEvolve, a framework that represents 3D scenes as executable code and improves reconstruction by evolving a harness around a small open‑source agent. SceneEvolve combines a compositional semantic action interface (SceneAct) with a harness evolution process (SceneHarness‑RSI) that uses a stronger teacher agent for iterative refinement. The resulting harness allows the small Qwen3.8‑27B model to reconstruct unseen scenes without parameter updates, achieving higher reconstruction quality and lower inference time than larger proprietary models across multiple datasets.
HARMONY is a hierarchical chain-of-thought framework that reconstructs complete 3D indoor scenes from a single monocular image. It combines agentic reasoning with visual geometry foundation models, starting with camera calibration and semantic layout recovery, then placing objects hierarchically while refining geometry with point cloud estimations. The method achieves semantically consistent scenes that align perceptually with the input image, outperforming existing baselines on synthetic and real-world data.
arXiv:2605. 10873v2 Announce Type: replace-cross Abstract: Recovering editable CAD programs from images or 3D observations is central to AI-assisted design, but progress is difficult to measure because existing evaluations are fragmented across datasets, modalities, and metrics.
WorldSculpt presents a method for generating compositional 3D representations of cluttered scenes with hundreds of objects by adapting a single-object 3D generative prior to multi-view observations. The approach, built on Pixal3D with a multi-view conditioning pathway, can generalize to highly occluded scenes without scene-level training. The authors also introduce the UE-MeshyScene benchmark and demonstrate that their method outperforms prior approaches across various evaluation settings, including converting existing 3DGS worlds into compositional mesh scenes.
arXiv:2606. 08402v2 Announce Type: replace-cross Abstract: Generating complete 3D scenes from a single image requires inferring globally consistent geometry, object relationships, and environmental context from inherently ambiguous visual evidence.