arXiv:2609.38615v1 Announce Type: cross Abstract: Egocentric videos of human manipulation provide valuable visual experience for embodied intelligence, yet collecting such data at scale is costly. Ex...
arXiv:2609.08636v1 Announce Type: cross Abstract: Egocentric 4D interaction forecasting aims to anticipate both where future interactions will occur in 3D and how the human body will move to realize...
Hand-object interaction (HOI) recognition requires capturing both hand manipulations and object transformations. However, existing video-language models often fall into shortcuts by relying on spurious correlations among hands, objects, or environmental context, rather than reasoning from the appearance and dynamics of hands and objects themselves.
UniRAP is a unified model that infers part-level physical affordances and translates language instructions into actionable geometric representations. It treats the task as a conditional multimodal generation problem, integrating visual inputs, textual instructions, and optional prompts into a shared spatiotemporal representation via a unified interface token mechanism. The model employs a Unified Affordance Decoder to jointly perform object detection, part-level affordance segmentation, and 4‑DoF interaction pose estimation, and uses a curriculum-based transfer training strategy to improve data efficiency with limited textual annotations.
arXiv:2610.27946v1 Announce Type: new Abstract: Egocentric action transfer aims to decouple the semantic action from egocentric observations and reproduce them in new visual contexts, enabling scalab...
ForeAct3D is a framework that grounds future world modeling in Vision‑Language‑Action policies by decoding depth, semantic segmentation, and camera pose from policy representations into current and future 3D scene states. The future queries are conditioned on the policy‑generated action chunk, and a physical‑consistency closure enforces background staticity, instance‑level rigidity, and wrist‑camera pose alignment with end‑effector kinematics. This approach eliminates the need for future prediction at inference and achieves high success rates on LIBERO and CALVIN, outperforming the base policy across all suites.
arXiv:2608.05369v2 Announce Type: replace-cross Abstract: Vision-language-action (VLA) models often treat main-view and wrist-view observations as parallel visual inputs, overlooking their distinct r...
arXiv:2610.02840v1 Announce Type: cross Abstract: World action models jointly learn to forecast world dynamics and predict robot actions, such that the learned internal world dynamics guide accurate...
The paper introduces P³-WAM, a unified world action model for egocentric dexterous manipulation that integrates 3D perception, action-conditioned prediction, and goal-directed planning. It operates in three modes—Perceive, Predict, and Plan—using a mixture-of-transformers architecture that shares base weights and adapts to modality-specific inputs. Experiments demonstrate that unified training improves performance across all three capabilities, outperforming specialized baselines on 3D hand reconstruction and action-conditioned video prediction while producing more instruction‑adherent manipulation videos.
The paper introduces HAP, a Hand-Driven Active Perception framework that predicts future six‑degree‑of‑freedom head motion in egocentric settings by conditioning on observed hand motion and inferred target context. HAP constructs a Predictive Target‑Centric Amodal Occlusion Graph to model current and potential occlusions among candidate objects, fuses this with hand and head motion history, and blends the learned trajectory with a constant‑velocity prior. Experiments on a public dataset and a newly released Bottle RGB‑D dataset demonstrate that HAP outperforms baseline methods in head‑motion prediction, highlighting the importance of hand‑driven intention and dynamic occlusion reasoning.
arXiv:2606.18558v2 Announce Type: replace Abstract: Motion forecasting is central to visual intelligence: agents must anticipate how objects will move in order to plan actions, reason about physical...
arXiv:2608.22449v1 Announce Type: cross Abstract: Forecasting dexterous hand motions from egocentric observations is fundamental to intelligent interactive systems. Existing VLM-based methods typical...