MAMHOI introduces an affordance-mediated factorization for generating scene-aware human‑object interactions. It separates scene understanding and motion synthesis: a scene‑conditioned model predicts feasible interaction locations and modes, while an affordance‑conditioned model generates the corresponding human‑object motion. This approach enables learning from separate human‑scene and human‑object datasets, reducing object‑scene penetration and improving interaction realism in complex indoor environments.
arXiv:2609.38615v1 Announce Type: cross Abstract: Egocentric videos of human manipulation provide valuable visual experience for embodied intelligence, yet collecting such data at scale is costly. Ex...
arXiv:2609.19119v1 Announce Type: new Abstract: Human videos contain rich causal evidence for robot manipulation: they reveal how hand motion induces object motion and produces task-relevant changes...
arXiv:2609.22332v1 Announce Type: cross Abstract: Generalizable robot manipulation requires predicting how a scene will evolve, identifying where interactions are feasible, and determining how to act...
UniRAP is a unified model that infers part-level physical affordances and translates language instructions into actionable geometric representations. It treats the task as a conditional multimodal generation problem, integrating visual inputs, textual instructions, and optional prompts into a shared spatiotemporal representation via a unified interface token mechanism. The model employs a Unified Affordance Decoder to jointly perform object detection, part-level affordance segmentation, and 4‑DoF interaction pose estimation, and uses a curriculum-based transfer training strategy to improve data efficiency with limited textual annotations.
Long-horizon human-object-scene interaction (HOSI) is central to embodied AI, animation, and game development. Existing human-object interaction (HOI) and human-scene interaction (HSI) models excel in...
World2HOSI is a multi‑expert harness that generates long‑horizon human‑object‑scene interaction (HOSI) data by orchestrating existing human‑object interaction (HOI) and human‑scene interaction (HSI) models. It builds a persistent scene state from decoupled background and object assets, plans action stages, invokes experts via their native interfaces, and updates the scene after each action, ensuring physical plausibility through bounded residual optimization and contact‑driven physics simulation. Experiments and a perceptual user study demonstrate that World2HOSI achieves superior task completion and interaction quality while producing more diverse data than prior approaches.
Vision-language perception has achieved impressive progress in aligning natural language with visual observations, yet grounding high-level semantics into part-level physical interaction remains chall...
arXiv:2606. 28813v1 Announce Type: cross Abstract: Human videos are a scalable source of supervision for robot manipulation, as they are abundant and naturally capture rich object interactions.
arXiv:2610.27066v1 Announce Type: new Abstract: Generating human-object interactions (HOIs) with multiple objects is challenging because the resulting motions arise from different underlying causes....
arXiv:2604. 01001v2 Announce Type: replace-cross Abstract: We introduce EgoSim, a closed-loop egocentric world simulator that generates spatially consistent interaction videos and persistently updates the underlying 3D scene state for continuous simulation.
The paper introduces MoMa-SG, a framework that constructs semantic‑kinematic 3D scene graphs for articulated scenes using RGB‑D sequences. It segments interactions, tracks points robustly, estimates joint parameters in a single optimization, and associates objects with articulations to detect contained items. The authors also present the Arti4D‑Semantic dataset with 62 real‑world sequences and demonstrate the system’s effectiveness on a quadruped and a mobile manipulator.
arXiv:2609.38466v1 Announce Type: new Abstract: Text-conditioned full-body human-object interaction (HOI) generation requires synthesizing human motion and object trajectories that match the input te...