← Back to all news
arXiv AI By Yiming Wang, Yonghao Dang, Huilai Li, Jiawei Tu, Jianqin Yin

Identity-Aware Human-Object Interaction Motion Captioning

Read the original on arXiv AI →

The paper introduces the Identity-Aware Human-Object Interaction Motion Captioning task, which requires captions to include both the subject’s identity and the interaction motion, e.g., "Sub_ID lifts the chair" instead of a generic description. It proposes ID‑HOINet, a model that learns from multi‑view videos using a Multi‑View Identity‑Motion Learning Module and a Two‑Stage Caption Rewriting Strategy to generate identity‑aware captions. Experiments show that ID‑HOINet achieves state‑of‑the‑art performance on the BEHAVE and InterCap datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
Sep 14

Uni-HOI:A Unified framework for Learning the Joint distribution of Text and Human-Object Interaction

Uni-HOI is a unified framework that learns the joint distribution among text, human motion, and object motion for 4D human‑object interaction (HOI). It uses large language models and two motion‑specific VQ‑VAEs to convert heterogeneous motion data into token sequences, enabling seamless integration of all three modalities. A two‑stage training strategy first captures correlations on a large‑scale HOI dataset and then fine‑tunes for specific tasks, achieving strong performance on text‑driven HOI generation, object‑motion‑driven human motion generation, and human‑motion‑driven object motion prediction.

By Mengfei Zhang, Jinlu Zhang, Zhigang Tu
llmscomputer-visionefficiency
More like this →
arXiv AI
2d ago

SituHOI: Scalable Capture of Scene-Situated Full-Body Human-Object Interaction

SituHOI is a large‑scale RGB‑D dataset that captures full‑body human‑object interactions in 16 indoor rooms, comprising 18,429 annotated clips and about 86 hours of activity. The dataset pairs moving‑camera recordings with static scene scans, object geometry, language descriptions, and estimated human and object trajectories, all registered in a shared floor‑aligned frame via a pipeline that integrates camera localization, depth‑based alignment, human mesh recovery, and object tracking. The data enable the FlexHOI‑Bench benchmark for goal‑conditioned joint motion prediction, and a reference predictor, SharedHOI, demonstrates strong performance on world‑space accuracy and human‑object coordination. whyItMatters":"The dataset provides scalable, scene‑aligned supervision for modeling human‑object interaction, addressing a key limitation of existing datasets and enabling more accurate motion prediction benchmarks."

By Qifan Wang, Qihang Fang, Boao Shi, Yushi Du, Pei Zhou, Siyan Dong, Yanchao Yang
benchmarkssafety
More like this →