Pose-Anchored Optical Flow for Low-Latency Human Action Anticipation in Human-Robot Teaming proposes PoseOFF, a representation that captures local motion around human joints by conditioning optical flow extraction on pose. This structured motion representation aligns with human kinematics and improves early action recognition accuracy across multiple datasets and backbones. PoseOFF achieves comparable or better performance while observing less of the action sequence, making it suitable for real‑time, resource‑constrained robotic systems.
By Lewis de Zoete Grundy, Chris McCarthy, Christopher Fluke
arXiv:2602. 19710v3 Announce Type: replace-cross Abstract: Existing Vision-Language-Action (VLA) models often suffer from feature collapse and low training efficiency because they entangle high-level perception with sparse, embodiment-specific action supervision.
By Haitao Lin, Hanyang Yu, Jingshun Huang, He Zhang, Yonggen Ling, Ping Tan, Xiangyang Xue, Yanwei Fu
arXiv:2608. 16222v1 Announce Type: cross Abstract: Humanoid intelligence requires learning over an extremely diverse space of whole-body motions and physically grounded interactions.
By Jiahao Ji, Ji Ma, Runhan Zhang, Runyi Yu, Wenjia Wang, Weiheng Chi, Qianqian Peng, Weichao Yan, Yongfei Gu, Ye Tian, Ting Wu, Longwei Li, Chun Yuan, Ruoli Dai, Lei Han
PACT is an end‑to‑end model that jointly learns human pose, contacts, and contact forces from monocular video. It augments a human reconstruction foundation model with learnable contact‑force tokens and a temporal transformer, and uses physics‑based supervision to enforce consistency between motion and forces. The authors also create a data annotation pipeline and a real‑world climbing benchmark, ForceWall, to train and evaluate the system, achieving state‑of‑the‑art performance and better generalization than staged approaches.
By Rikhat Akizhanov (MBZUAI), Yangsong Zhang (MBZUAI), Nikolai Kaliazin (MBZUAI), Peter Wolf (ETH Z\"urich), Yoshihiko Nakamura (MBZUAI), Pascal Fua (EPFL), Fabio Pizzati (MBZUAI), Ivan Laptev (MBZUAI)
SocioGesture is a real‑time, adaptive system for recognizing social gestures in human‑robot interaction. It employs a compact, confidence‑aware body‑hand skeleton representation and a lightweight dual‑stream model that fuses body motion with hand articulation, enabling low‑latency onboard recognition. The model is trained with occlusion‑aware skeleton corruption to handle missing hands, occluded arms, and unstable keypoints, and it can expand its gesture vocabulary during deployment by saving uncertain interaction segments for offline labeling.
By Wenjin Fu, Li-Fan Wu, Jerin Peter, Chip Huyen, Boyuan Chen, Jan Liphardt
arXiv:2609.19119v1 Announce Type: new
Abstract: Human videos contain rich causal evidence for robot manipulation: they reveal how hand motion induces object motion and produces task-relevant changes...
By Jiaming Zhang, Homanga Bharadhwaj
arXiv:2609.08636v1 Announce Type: cross
Abstract: Egocentric 4D interaction forecasting aims to anticipate both where future interactions will occur in 3D and how the human body will move to realize...
By Qiaohui Chu, Haoyu Zhang, Meng Liu, Haoxiang Shi, Dongmei Jiang, Liqiang Nie
BeyondRetarget is an end‑to‑end framework that learns to generate executable humanoid robot motions directly from monocular RGB videos, bypassing the need for an explicit human motion representation. By learning robot‑oriented implicit representations and incorporating a contact‑aware motion optimization mechanism, the method captures cross‑morphology motion structures and improves temporal consistency and physical plausibility. Experiments demonstrate that BeyondRetarget achieves higher execution success rates, lower latency, and greater accuracy and robustness in both simulation and real humanoid robots.
By Tianyu Xiong, Yi Lu, Jinrui Wang, Ziqi Liang, Dandan Lei, Xiaoyang Zhou, Xiao-xiao Long, Qiu Shen, Xun Cao
Being‑M0.7 is a latent world‑action model that learns from a large mixed‑modality human dataset (over 10,000 hours of video‑only, motion‑only, and paired video‑motion data) to predict future latent visual states and motion. The model is pre‑trained on human data, adapted to robot viewpoints during mid‑training, and then combined with an action expert via gated cross‑attention during post‑training to produce executable whole‑body commands. In experiments, Being‑M0.7 outperforms baselines on the SIMPLE benchmark and matches the best baseline on real‑world Unitree G1 loco‑manipulation tasks.
By Junpeng Yue, Boyuan Li, Yuxuan Wang, Zepeng Wang, Yuhui Fu, Feiyang Xie, Yu Zhang, Jing Zhang, Xianqi Zhang, Weibo Li, Xiaofei Zheng, Yuming Fang, Jiangxing Wang, Zongqing Lu
World models endow perceptual systems with the ability to predict how scenes evolve under interaction. They are most beneficial when trained on diverse volumes of data, to instill a rich prior into do...
The paper introduces Skeleton-Language feature Pooling Switching, a weakly‑supervised vision‑language pretraining strategy for skeleton‑based zero‑shot spatio‑temporal action localization. It replaces video‑level pooling with instance‑level feature computation during inference, enabling the model to estimate unseen actions without costly annotations. Additionally, Scene‑Mixed Discriminative Contrastive Learning is proposed to separate actions at the instance level within mixed scenes using a MIL framework, and experiments on four public datasets confirm the method’s effectiveness.
By Koshiro Nagano, Fumiaki Sato, Ryo Hachiuma, Kazuki Tsutsukawa, Taiki Sekii
PhysBrain 1.5 is a unified vision‑language model that learns to understand physical environments, generate actions, and predict future states by encoding language, end‑effector motion, and dense visual targets as discrete sequences and training them with autoregressive next‑token prediction. The model is pre‑trained on human interaction videos and fine‑tuned on human demonstrations, robot trajectories, and simulated experience, achieving an average score of 72.5 across 28 embodied understanding benchmarks and outperforming other open‑source models on 14 of them. It also demonstrates the ability to produce end‑effector trajectories and predict future scenes with spatially aligned RGB, depth, and robot‑mask outputs.
By DeepCybo Team, Yu Bin, Haipeng Cao, Zheng Chang, Kai Chen, Youning Chen, Kailin Deng, Yichao Du, Xiaotong Fu, Haoyang Ge, Yunlong Guo, Chenliu Hao, Jiyan He, Xuguo He, Yakun Hou, Kai Hu, Cong Huang, Tuopusen Huang, Yu Huang, Hong Li, Peize Li, Shijie Lian, Xiaopeng Lin, Yun Lin, Haibao Liu, Haochen Liu, Qiuzhi Liu, Shengcai Liu, Zhiqiang Liu, Tao Luo, Peng Ren, Shuo Ren, Chaoyi Ruan, Zhaolong Shen, Yukun Shi, Qiyuan Su, Yuxuan Tian, Yining Wang, Changti Wu, Hao Wu, Xueyin Xu, Ruoqi Yang, Zhaoyang Yang, Hang Yuan, Zhaoyang Zeng, Hanwen Zhang, Ruimeng Zhang, Yao Zhang, Yibo Zhang, Yuxiang Zhang, Zhirui Zhang, Ziyi Zhang, Zubin Zheng, Zishen Zhuang