arXiv AI
Jul 10
EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data
arXiv:2607. 08436v1 Announce Type: cross Abstract: Egocentric human data offers scalable supervision for robot manipulation.
RGB-D Video Generation for Improving Human-to-Robot Object Handover Prediction
The paper introduces Hand2Bot, an RGB‑D video dataset designed for human‑to‑robot handover scenarios, capturing body posture and facial expressions amid real‑world noise. It also proposes PassGen, a generative pipeline using stable video diffusion and an Intention‑Aware Temporal Face Encoder to synthesize realistic handover sequences while maintaining hand‑object consistency. A morphology‑based depth editing strategy is employed to replicate realistic sensor noise, and experiments show that training on PassGen yields high intention identification accuracy, low false trigger rates, and robust zero‑shot transfer to a physical robot platform.
Related stories
arXiv Computer Vision
Sep 1
ACE-Ego-Hand: Repurposing Video Diffusion Models for Occlusion-Robust Egocentric 3D Hand Motion Recovery
arXiv:2608.20308v2 Announce Type: replace Abstract: Egocentric video offers scalable manipulation data for embodied AI, yet recovering metric 3D hand trajectories remains challenging due to severe ob...
arXiv Computer Vision
Aug 21
DreamHand: Repurposing Video Diffusion Models for Occlusion-Robust Egocentric 3D Hand Motion Recovery
arXiv:2608. 20308v1 Announce Type: new Abstract: Egocentric video offers scalable manipulation data for embodied AI, yet recovering metric 3D hand trajectories remains challenging due to severe object occlusion and frequent out-of-sight gaps.
Hugging Face Trending Papers
Aug 12
HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editing
Robotic manipulation with dexterous hands is a cornerstone of Embodied AI, yet its progress is stifled by the high cost of collecting embodiment-aware teleoperation data. While abundant egocentric videos of human hands offer a scalable alternative, the profound discrepancies in appearance, articulation, and camera viewpoints between human and robotic data raise significant challenges for co-training.
arXiv Machine Learning
Jun 4
PointAction: 3D Points as Universal Action Representations for Robot Control
arXiv:2606. 03943v1 Announce Type: cross Abstract: Video-Action Models (VAMs) leverage the broad visual dynamics captured by pre-trained video diffusion models, offering a promising path toward generalizable robot manipulation.
Hugging Face Trending Papers
Jul 13
HandFlow: Fully Generative 4D Hand Recovery with Flow Matching
Accurate monocular 4D hand reconstruction remains challenging. Per-frame discriminative regressors lack temporal context and often produce jittery predictions.
arXiv AI
Jul 14
HandFlow: Fully Generative 4D Hand Recovery with Flow Matching
arXiv:2607. 11221v1 Announce Type: cross Abstract: Accurate monocular 4D hand reconstruction remains challenging.
Hugging Face Trending Papers
Jul 8
GIRAF: Towards Generalizable Human Interactions with Articulated Objects
Synthesizing realistic full-body human interactions with articulated objects is a fundamental challenge for embodied AI and graphics, with applications in robotics training and virtual agents. Existing models remain limited: some focus on simple activities with static objects, while others restrict attention to hand-only manipulation.
arXiv Computer Vision
6d ago
An Evaluation Framework for Generating Multi-View Images of a Person in a Scene
The paper introduces a framework for generating multi‑view images of a person within a natural scene, addressing the scarcity of paired multi‑view datasets for human subjects. It evaluates existing diffusion‑based image‑editing models and finds they often hallucinate head‑turn angles, leading to inconsistent backgrounds. To overcome this, the authors propose the Head Scene Rotation Difference (HSRD) metric, which separates camera movement from head pose changes and enables reliable assessment of 3D spatial parallax for constructing high‑quality synthetic datasets.
arXiv Machine Learning
2d ago
HuRo: Robotizing Human Videos for Scalable VLA Pretraining
The paper introduces HuRo, a dataset of 630K robotized episodes derived from diverse human videos, created via a pipeline that aligns observations and actions for robotic use. Experiments on four real‑world manipulation tasks show that scaling robotized pretraining boosts task completion from 51.5% to 80.3% and improves out‑of‑distribution performance under spatial and visual shifts. Ablation studies reveal that visual robotization enhances robustness and that end‑to‑end pretraining with retargeted actions outperforms visual‑only transfer.
arXiv AI
Aug 11
X2C: A Dataset Featuring Nuanced Facial Expressions for Realistic Humanoid Imitation
arXiv:2505. 11146v3 Announce Type: replace-cross Abstract: Fine-grained facial expression transfer from humans to humanoid agents presents a unique pattern recognition challenge due to the significant domain gap between biological facial dynamics and mechanical control spaces.
arXiv AI
Jun 8
What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos?
arXiv:2606. 06627v1 Announce Type: cross Abstract: Human video datasets used for cotraining robot manipulation policies largely consist of curated demonstrations where motions are orchestrated to resemble robot behavior and 3D hand poses are captured with specialized hardware.