SignRR is a new sign language production framework that combines retrieval of real sign motion segments with a learned refinement step to produce globally coherent signing sequences. It starts from a dictionary of authentic sign segments and refines them using a part-aware Residual VQ‑VAE, preserving fine hand articulation while handling temporal length differences in latent space. Experiments on PHOENIX14T and CSL‑Daily demonstrate state‑of‑the‑art back‑translation performance and competitive pose quality.
By Fidel Omar Tito Cruz, Angie Sanchez Marquina, Summy Farfan, Gissella Bejarano
arXiv:2610.20124v1 Announce Type: new
Abstract: Human mesh recovery (HMR) provides a geometric foundation for understanding human motion, but accurately reconstructing both body motion and fine hand...
By Zhengjie Xu, Ruihao Gong, Xianglong Liu
arXiv:2609.19119v1 Announce Type: new
Abstract: Human videos contain rich causal evidence for robot manipulation: they reveal how hand motion induces object motion and produces task-relevant changes...
By Jiaming Zhang, Homanga Bharadhwaj
arXiv:2605. 01720v3 Announce Type: replace-cross Abstract: Existing large-scale sign language resources typically provide supervision only at the level of raw video-text alignment and are often produced in laboratory settings.
By Sen Fang, Hongbin Zhong, Yanxin Zhang, Dimitris N. Metaxas
arXiv:2609.14122v1 Announce Type: new
Abstract: We study the challenge of sign language video mimicking: given a driving video and a single reference frame, synthesize a video where the target signer...
By Zhewen He (New York University Abu Dhabi), Junyi Yu (New York University Abu Dhabi), Haomian Huang (New York University Abu Dhabi), Zhenhua Li (ChatSign Technology), Yi Fang (New York University Abu Dhabi, ChatSign Technology)
arXiv:2610.32517v1 Announce Type: new
Abstract: Hand landmarks serve as input for gesture recognition, sign language understanding, and touchless human-computer interaction. Yet hand pose estimators...
By Gibran Benitez-Garcia, Hiroki Takahashi
arXiv:2609.13269v1 Announce Type: cross
Abstract: Gesture recognition on video is normally posed as classification: label each frame, then act on the label. That is adequate for control, where a comm...
By Amey Thakur
Synthesizing realistic full-body human interactions with articulated objects is a fundamental challenge for embodied AI and graphics, with applications in robotics training and virtual agents. Existing models remain limited: some focus on simple activities with static objects, while others restrict attention to hand-only manipulation.
The paper introduces GALA, a Geometry-Aware Latent Action modeling framework that enhances image-based latent actions with 3D end‑effector motion. It proposes the Unified End‑effector Motion Representation (UEMR) to preserve fine‑grained motion while improving cross‑embodiment generalizability. Experiments show GALA effectively models generalizable fine‑grained motions across embodiments, achieving high success rates in RoboCasa-GR1 and real‑world tasks.
By Yichen Liu, Puzhen Yuan, Xiang Zhu, Yanjiang Guo, Jianyu Chen
SignDEG (DINO Eats Geometry) is a temporal sign language encoder that incorporates articulated hand geometry as a pre‑training target while using only video at inference. It leverages frozen DINOv3 features of tracked hands and faces, feeding them into per‑stream temporal Transformers, and introduces two geometric objectives—Geometric Latent Prediction (GLP) and Geometric Relation Transfer (GRT)—to supervise intermediate and final blocks. Jointly trained with temporal self‑distillation, SignDEG achieves higher BLEU‑4 scores on How2Sign/OpenASL and improves recall on ASL Citizen and Sem‑Lex datasets compared to SHuBERT, demonstrating that articulated structure can be transferred into an appearance encoder without requiring a hand reconstruction model at deployment.
By Junyi Hu, Zhewen He, Haomian Huang, Zhenhua Li, Zhifei Li, Shuaihang Yuan, Yi Fang
TouchéWorld is a framework that merges gesture-driven interaction with camera-controlled video world models, enabling users to explore generated environments using both camera navigation and continuous hand motion. Starting from an initial image and a scene-description prompt, the system generates video conditioned on temporally aligned camera and hand controls, built from joint camera–hand training data derived from heterogeneous egocentric videos. The approach incorporates spatial hand-skeleton conditioning, numerical hand-motion features, causal adaptation, and few-step distillation to support chunk-level online generation, while TouchéBench evaluates hand-motion following, camera–hand coordination, online interaction, and object response.
By Jiaao Yu, Yanjian Zhang, Haoyang Jiang, Guillaume Wisniewski, Nadi Tomeh, Shijie Jia
MANGO‑Grasp introduces an anisotropic interaction framework for cross‑embodiment dexterous grasping. It represents objects as geometry‑oriented 3D Gaussian primitives to preserve local surface geometry, encodes hands with pretrained descriptors that capture morphology and kinematics, and models hand‑object interaction using anisotropic Mahalanobis fields. The method achieves state‑of‑the‑art performance on CMAP and MultiGripperGrasp benchmarks, outperforms baselines by up to 8.24 percentage points in simulation, and demonstrates strong zero‑shot transfer to the unseen SharpaWave hand with up to 16.57 percentage point improvement, achieving 86% success in real‑world experiments.
By Heng Zhang, Kevin Yuchen Ma, Mike Zheng Shou, Weisi Lin, Yan Wu