EgoFun3D introduces a coordinated task, dataset, and benchmark for creating simulation-ready interactive 3D objects from egocentric videos. The approach captures general cross-part functional mappings via function templates, enabling precise evaluation and direct compilation into executable code. A 4‑stage pipeline—2D part segmentation, reconstruction, articulation estimation, and function template inference—is proposed, and a dataset of 517 videos with detailed annotations is released for comprehensive benchmarking.
arXiv:2604. 01001v2 Announce Type: replace-cross Abstract: We introduce EgoSim, a closed-loop egocentric world simulator that generates spatially consistent interaction videos and persistently updates the underlying 3D scene state for continuous simulation.
Ego2Act is a new benchmark that tests video generation models on goal‑directed, egocentric manipulation tasks. It contains 2,640 videos from 110 real‑world tasks, each requiring multiple steps of object manipulation to achieve a high‑level goal. The benchmark includes Ego2ActJudge, a reference‑free evaluation pipeline that better aligns with human judgments of task completion and physics plausibility.
arXiv:2603. 09731v3 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) are increasingly considered as a foundation for embodied agents, yet it remains unclear whether they can reliably reason about the long-term physical consequences of actions from an egocentric viewpoint.
FAMOS is a feed‑forward model that predicts movable‑part segmentation and joint parameters from a sparse, unordered set of partial point clouds. It jointly reasons over multiple observations using a Multi‑state Articulation Transformer that alternates state‑wise and global attention, and introduces an observed articulation span objective to supervise motion ranges across inputs. A procedural data generator supplies self‑annotated assets for training, and experiments on PartNet‑Mobility, ACD, and ArtiCraft‑10K show consistent improvements over existing feed‑forward and optimization‑based baselines.
arXiv:2603.20169v2 Announce Type: replace Abstract: Generative world models have shown promise for simulating dynamic environments, yet egocentric video remains challenging due to rapid viewpoint cha...
arXiv:2603. 16085v2 Announce Type: replace-cross Abstract: Recent breakthroughs in 3D generation have enabled the synthesis of high-fidelity individual assets.
arXiv:2609.39378v1 Announce Type: new Abstract: Real-world embodied tasks, from everyday activities to professional procedures, require agents to act under physical constraints while tracking evolvin...
arXiv:2606. 16202v1 Announce Type: cross Abstract: Humans naturally understand object physics through everyday interactions, but faithfully predicting complex deformable dynamics, such as elastic materials and fabrics, remains a major challenge for computer vision and robotics.
Grounded-Exo2Ego introduces a dual‑branch video diffusion model that combines a geometric anchoring branch with a semantic grounding branch to generate egocentric video from a single exocentric source. The framework also includes a camera re‑localization algorithm to correct reconstruction misalignment and a fully automated synthetic data engine for training. Experiments on the EgoExo4D dataset demonstrate significant performance gains over recent state‑of‑the‑art methods.
WorldSculpt presents a method for generating compositional 3D representations of cluttered scenes with hundreds of objects by adapting a single-object 3D generative prior to multi-view observations. The approach, built on Pixal3D with a multi-view conditioning pathway, can generalize to highly occluded scenes without scene-level training. The authors also introduce the UE-MeshyScene benchmark and demonstrate that their method outperforms prior approaches across various evaluation settings, including converting existing 3DGS worlds into compositional mesh scenes.
MINT is a foundation model that directly predicts world-space two-hand trajectories from egocentric RGB video, jointly estimating camera motion, hand states, and hand presence in a single spatiotemporal representation. It uses an open-source labeling pipeline, EGOPIPELINE, to generate large-scale pseudo-labels for pretraining, followed by fine-tuning on a small set of high-quality joint annotations. The model outperforms existing multi-stage approaches in accuracy and speed, and generalizes zero‑shot to unseen egocentric datasets.
arXiv:2609.01276v1 Announce Type: new Abstract: Complete 3D perception from egocentric video requires recovering the surrounding scene and the wearer's full-body motion in a shared metric frame. Exis...