← Back to all news
arXiv Computation and Language By Junyi Hu, Zhewen He, Haomian Huang, Zhenhua Li, Zhifei Li, Shuaihang Yuan, Yi Fang

DINO Eats Geometry: Self-Supervised Sign Representations that Absorb Articulated Hand Geometry

Read the original on arXiv Computation and Language →

SignDEG (DINO Eats Geometry) is a temporal sign language encoder that incorporates articulated hand geometry as a pre‑training target while using only video at inference. It leverages frozen DINOv3 features of tracked hands and faces, feeding them into per‑stream temporal Transformers, and introduces two geometric objectives—Geometric Latent Prediction (GLP) and Geometric Relation Transfer (GRT)—to supervise intermediate and final blocks. Jointly trained with temporal self‑distillation, SignDEG achieves higher BLEU‑4 scores on How2Sign/OpenASL and improves recall on ASL Citizen and Sem‑Lex datasets compared to SHuBERT, demonstrating that articulated structure can be transferred into an appearance encoder without requiring a hand reconstruction model at deployment.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computer Vision
Sep 15

SignMimic: Robust High-Quality Sign Language Motion Generation via Human-Shape-Oblivious Pose Transfer Guidance

arXiv:2609.14122v1 Announce Type: new Abstract: We study the challenge of sign language video mimicking: given a driving video and a single reference frame, synthesize a video where the target signer...

By Zhewen He (New York University Abu Dhabi), Junyi Yu (New York University Abu Dhabi), Haomian Huang (New York University Abu Dhabi), Zhenhua Li (ChatSign Technology), Yi Fang (New York University Abu Dhabi, ChatSign Technology)
diffusionbenchmarks
More like this →
arXiv Computer Vision
Sep 3

SignMatch: Matching Dictionary Signs to Continuous Sign Language Video

SignMatch introduces a prototype‑structured embedding space that learns to match dictionary sign videos with continuous sign language footage based solely on visual similarity of handshape and motion. By mapping isolated dictionary exemplars into this space, the method enables direct, embedding‑based sign matching and can generalise to unseen signs using only dictionary examples. Experiments on ASL‑Citizen, ChaLearn OSLWL, and BOBSL CSLR2 benchmarks show strong cross‑dataset, cross‑task, and cross‑language performance, outperforming prior approaches on American, British, and Spanish sign languages without benchmark‑specific supervision.

By Ryan Wong, Youngjoon Jang, Liliane Momeni, G\"ul Varol, Andrew Zisserman
ragroboticsbenchmarks
More like this →
arXiv Computer Vision
Aug 31

SignRR: Retrieve and Refine Real Motion for Sign Language Production

SignRR is a new sign language production framework that combines retrieval of real sign motion segments with a learned refinement step to produce globally coherent signing sequences. It starts from a dictionary of authentic sign segments and refines them using a part-aware Residual VQ‑VAE, preserving fine hand articulation while handling temporal length differences in latent space. Experiments on PHOENIX14T and CSL‑Daily demonstrate state‑of‑the‑art back‑translation performance and competitive pose quality.

By Fidel Omar Tito Cruz, Angie Sanchez Marquina, Summy Farfan, Gissella Bejarano
diffusionefficiencybenchmarks
More like this →