arXiv:2609.38466v1 Announce Type: new Abstract: Text-conditioned full-body human-object interaction (HOI) generation requires synthesizing human motion and object trajectories that match the input te...
Synthesizing realistic Human-Object Interactions (HOI) is critical for creating embodied avatars and functional virtual environments. However, current data-driven approaches primarily rely on motion capture datasets, which are expensive to scale and limited in functional diversity.
Uni-HOI is a unified framework that learns the joint distribution among text, human motion, and object motion for 4D human‑object interaction (HOI). It uses large language models and two motion‑specific VQ‑VAEs to convert heterogeneous motion data into token sequences, enabling seamless integration of all three modalities. A two‑stage training strategy first captures correlations on a large‑scale HOI dataset and then fine‑tunes for specific tasks, achieving strong performance on text‑driven HOI generation, object‑motion‑driven human motion generation, and human‑motion‑driven object motion prediction.
MAMHOI introduces an affordance-mediated factorization for generating scene-aware human‑object interactions. It separates scene understanding and motion synthesis: a scene‑conditioned model predicts feasible interaction locations and modes, while an affordance‑conditioned model generates the corresponding human‑object motion. This approach enables learning from separate human‑scene and human‑object datasets, reducing object‑scene penetration and improving interaction realism in complex indoor environments.
Long-horizon human-object-scene interaction (HOSI) is central to embodied AI, animation, and game development. Existing human-object interaction (HOI) and human-scene interaction (HSI) models excel in...
SMART introduces a scalable system for zero‑shot sim‑to‑real articulated object manipulation by leveraging large‑scale synthetic pretraining. The core component, SMART‑Sim, is an articulation‑aware simulation platform that facilitates efficient task generation and demonstration collection. Using SMART‑Sim, the authors synthesize SMART‑Data, comprising over 1 million demonstrations across 44 task types, 5 robot setups, and 2,507 articulated objects, and demonstrate that a vision‑language‑action model pretrained on this data achieves competitive simulation performance and zero‑shot transfer to real‑world tasks.
arXiv:2609.32551v2 Announce Type: replace-cross Abstract: Text-conditioned human-object interaction (HOI) generation requires body motion, object trajectories & rotations, and hand articulation to re...
World2HOSI is a multi‑expert harness that generates long‑horizon human‑object‑scene interaction (HOSI) data by orchestrating existing human‑object interaction (HOI) and human‑scene interaction (HSI) models. It builds a persistent scene state from decoupled background and object assets, plans action stages, invokes experts via their native interfaces, and updates the scene after each action, ensuring physical plausibility through bounded residual optimization and contact‑driven physics simulation. Experiments and a perceptual user study demonstrate that World2HOSI achieves superior task completion and interaction quality while producing more diverse data than prior approaches.
arXiv:2609.22611v1 Announce Type: cross Abstract: Humanoid robots can acquire complex skills by imitating kinematic humanoid motion references, yet reliable references for contact-rich interactions r...
Synthesizing realistic full-body human interactions with articulated objects is a fundamental challenge for embodied AI and graphics, with applications in robotics training and virtual agents. Existing models remain limited: some focus on simple activities with static objects, while others restrict attention to hand-only manipulation.
arXiv:2609.36454v1 Announce Type: new Abstract: We study hand-object interaction (HOI) reconstruction from monocular RGB videos, where partial observations can produce visually plausible yet mechanic...
DeVI (Dexterous Video Imitation) is a framework that learns physics‑based character control for dexterous human‑object interaction by imitating text‑conditioned synthetic videos. It introduces hybrid imitation targets that combine reconstructed 3D human motion with tracked 2D object trajectories, avoiding explicit 3D object motion reconstruction. Visual HOI Alignment further refines the human reference to match both the generated video and the initial 3D object configuration, improving reconstruction quality and imitation success.