arXiv Computer Vision By Suozhi Huang, Kelsey Fu, Binxu Li, Luna Lyu, Kaize Wu, Wenbo Lu, David Smerkous, Jiahan Li, Siqi Zhu, Zhizhou Sha, Jiacheng Guo, Zhixuan Liang, Shilong Liu, Le Cong, Mengdi Wang

LabWorld: Learning Interactive 4D Worlds for Scientific Experimentation

Read the original on arXiv Computer Vision →

LabWorld is a Real2Sim framework that reconstructs interactive 4D laboratory worlds from monocular RGB video, producing reusable 3D assets, metric scenes, time‑varying object states, and procedure events. By leveraging multi‑frame observations and motion reconstruction that combines visual, geometric, and event constraints, LabWorld outperforms existing baselines on real pipetting and vortexing tasks, achieving higher Mask IoU and PCK‑AUC scores. The framework enables explicit, editable state representation for replay, inspection, and future embodied scientific systems.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI
Oct 27

SceneAgent: An Agentic Pipeline from 3D and 4D Captures to Simulation Environments

SceneAgent is an agentic pipeline that transforms 3D and 4D visual captures—such as 3D Gaussian Splatting, photogrammetry, LiDAR, AI‑generated scenes, and dynamic 4D videos—into interactive simulation environments. It coordinates object reconstruction, physical modeling, part decomposition, articulation, and deformable simulation, and then generates diverse object variants and scripted demonstrations to support policy learning. In benchmark tests, SceneAgent outperforms SimFoundry on cluttered bin scenes and dramatically improves dynamic IoU on 4DCodeBench, while also enabling simulated interactions and preliminary real‑robot policy transfer.

By Luke Hollis, Tianxing Fan, Heng Yang
arXiv Computer Vision
Sep 7

TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Image

TourPhysics is an online framework that builds physics‑grounded visual world models from a single image and a declarative physical configuration. It integrates deterministic simulation with video generation, separating simulator state, geometric evidence, generator controls, and appearance memory to produce consistent observations for exploration and manipulation. The system preserves the input scene, follows prescribed camera and object trajectories more closely than baselines, and reduces appearance drift during long‑horizon revisits.

By Xin Zhang, Yabo Chen, Zixuan Duan, Haibin Huang, Chi Zhang, Feng Xu, Xuelong Li
arXiv AI
Jun 12

An Embodied Simulation Platform, Benchmark, and Data-Efficient Augmentation Framework for Wet-Lab Robotics

arXiv:2606. 12936v1 Announce Type: cross Abstract: Wet-lab robots can improve the reproducibility, throughput, and safety of biomedical experiments, but scaling their learning requires customizable simulators for safe and reproducible task generation, open editable laboratory assets, and efficient pipelines that turn limited demonstrations into usable training data.

By Zhe Liu, Huanbo Jin, Zhaohui Du, Zhe Wang, He Xu, Peijia Li, Jiaming Gu, Quan Lu, Qi Wang, Bin Ji, Ting Xiao
arXiv Computer Vision
Aug 27

4DStreamCtrl: Interactive Video Generation with Online 4D Control

The paper introduces 4DStreamCtrl, a system that unifies camera motion, object trajectories, and depth into a single 3D point‑track representation, enabling joint control, depth editing, and motion transfer in a single forward pass. By mining in‑the‑wild video for 3D motion supervision and encoding it with a lightweight Geometric Motion Head, the authors train a causal streaming student that can generate arbitrarily long videos in just four denoising steps, achieving 20 FPS on a single high‑end GPU for 480p video. This approach outperforms prior camera‑only, 2D, and offline‑3D methods in motion‑control precision while maintaining temporal coherence over hundreds of frames, thereby enabling interactive 4D‑controllable streaming generation for the first time.

By Shiqian Li, Chenguo Lin, Zhiguang Liu, Yu Tang, Jiarong Ou, Rui Chen, Yixin Zhu