MessyKitchens introduces a new dataset of cluttered real-world kitchen scenes with detailed 3D object shapes, poses, and accurate contact information. The authors extend the SAM 3D single-object reconstruction method with a Multi-Object Decoder (MOD) to jointly reconstruct entire scenes, achieving better registration accuracy and reduced inter-object penetration compared to prior work. The dataset, benchmark, code, and pretrained models will be publicly released on the project website.
By Junaid Ahmed Ansari, Ran Ding, Fabio Pizzati, Ivan Laptev
CARI4D is a category‑agnostic method that reconstructs spatially and temporally consistent 4D human‑object interactions from monocular RGB videos at metric scale. It introduces a pose hypothesis selection algorithm that integrates predictions from foundation models, refines them via a learned render‑and‑compare paradigm, and reasons about contacts to satisfy physical constraints. Experiments demonstrate a 38% improvement on in‑distribution data and 36% on unseen data compared to prior work, and the model generalizes to unseen categories, enabling zero‑shot application to internet videos.
By Xianghui Xie, Bowen Wen, Yan Chang, Hesam Rabeti, Jiefeng Li, Ye Yuan, Gerard Pons-Moll, Stan Birchfield
arXiv:2609.19119v1 Announce Type: new
Abstract: Human videos contain rich causal evidence for robot manipulation: they reveal how hand motion induces object motion and produces task-relevant changes...
By Jiaming Zhang, Homanga Bharadhwaj
Human mesh recovery (HMR) aims to recover 3D human meshes from images. Most existing HMR benchmarks and methods focus on either multi-person reconstruction from a single view or single-person reconstruction from multiple views, where the number of subjects and the scene scale are relatively limited.
arXiv:2511. 16624v2 Announce Type: replace-cross Abstract: We present SAM 3D, a generative model for visually grounded 3D object reconstruction, predicting geometry, texture, and layout from a single image.
By SAM 3D Team, Xingyu Chen, Fu-Jen Chu, Pierre Gleize, Kevin J Liang, Alexander Sax, Hao Tang, Weiyao Wang, Michelle Guo, Thibaut Hardin, Xiang Li, Aohan Lin, Jiawei Liu, Ziqi Ma, Anushka Sagar, Bowen Song, Xiaodong Wang, Jianing Yang, Bowen Zhang, Piotr Doll\'ar, Georgia Gkioxari, Matt Feiszli, Jitendra Malik
arXiv:2605.20992v4 Announce Type: replace
Abstract: We ask whether everyday open-world monocular videos can be turned into reusable 4D interaction primitives: articulated hand motion, object shape wi...
By Hao Xu, Yilin Liu, Yinqiao Wang, Chi-Wing Fu, Niloy J. Mitra
arXiv:2606. 28215v1 Announce Type: cross Abstract: Extracting dynamic 4D object interactions from massive, in-the-wild monocular videos offers a highly efficient data collection pathway for scaling Embodied AI and training VLAs.
By Jiaxin Li, Yuxiang Wu, Zhenkai Zhang, Xinrui Shi, Haoyuan Wang, Yichen Zhao, Su Linxiang, Chenyang Yu, Mingyu Zhang, Yifan Ding, Boran Wen, Li Zhang, Ruiyang Liu, Yong-Lu Li
The paper introduces a top‑down approach for multi‑person 3D reconstruction from multiple views, using a unified, instance‑centric human‑aware 3D space. Observations from different cameras are lifted into this shared space where geometry, appearance, and semantic cues are jointly encoded, and a spatial contrastive learning strategy aligns features of the same person across views while separating different individuals. The method then regresses SMPL parameters from 3D tokens in a feed‑forward manner, achieving robust, accurate, and efficient reconstruction even under severe occlusions.
By Yuanwang Yang, Buzhen Huang, Zongxuan Ren, Jing Huang, Kun Li
arXiv:2602. 08058v3 Announce Type: replace-cross Abstract: In the presence of occlusions and measurement noise, geometrically accurate scene reconstructions -- which fit the sensor data -- can still be physically incorrect.
By Xihang Yu, Rajat Talak, Lorenzo Shaikewitz, Luca Carlone
HORIA presents a method for jointly reconstructing a human and an object in 4D from a single monocular video. It tackles two main challenges—complex, multi‑phase interaction semantics and occlusion‑induced incomplete views—by first segmenting the video into temporally grounded, motion‑aware interaction phases with text captions, then using these captions to guide an interaction‑aware re‑rendering that refines the reconstruction with novel‑view supervision from a text‑to‑video diffusion prior. The resulting framework delivers temporally coherent, accurate 4D reconstructions across diverse interaction scenarios, outperforming existing approaches.
By Hyeongjin Nam, Taeryung Lee, Daniel Sungho Jung, Kyoung Mu Lee
arXiv:2603.12789v3 Announce Type: replace
Abstract: Recent advances in 3D foundation models have enabled joint reconstruction of humans and their surrounding environments. However, combining independ...
By Sangmin Kim, Minhyuk Hwang, Geonho Cha, Dongyoon Wee, Jaesik Park
arXiv:2610.10539v1 Announce Type: new
Abstract: We propose Tetris3D, a generative framework for single-image 3D scene reconstruction that recovers objects which are physically and geometrically coher...
By Jaeyeong Kim, Jinhyuk Jang, Jongmin Lee, Kyehong Park, Seungryong Kim