arXiv AI
Aug 10
TransSLR: A Lightweight Transformer for Sign Language Recognition
arXiv:2608. 06407v1 Announce Type: cross Abstract: Automated Sign Language Recognition for under-represented languages remains a largely unsolved problem.
A Hybrid Kinematic GCN and Elastic Transformer for Efficient Indian Sign Language Recognition
Related stories
arXiv Computer Vision
Aug 26
Stack Transformer Based Spatial-Temporal Attention Model for Dynamic Sign Language and Fingerspelling Recognition
The paper introduces the Sequential Spatio-Temporal Attention Network (SSTAN), a Transformer-based architecture that replaces traditional Graph Convolutional Networks for sign language recognition. SSTAN uses a hierarchical, stacked design with Spatial Multi-Head Attention to model joint relationships within frames and Temporal Multi-Head Attention to capture long-range dependencies across frames, eliminating the need for predefined skeletal graphs. Experiments on large-scale datasets (WLASL, JSL, KSL) show that SSTAN, trained from scratch, achieves state‑of‑the‑art performance in fingerspelling categories and outperforms other skeleton‑only methods on WLASL, highlighting its data efficiency and ability to learn complex spatio‑temporal patterns.
arXiv Computer Vision
Aug 27
SMART: MLLM-guided Temporal Alignment for Unifying Sign Language Recognition and Spotting
SMART is a new framework that jointly tackles continuous sign language recognition (CSLR) and spotting by leveraging a multimodal large language model (MLLM) to generate motion descriptions as auxiliary semantic cues. It performs stable video‑text alignment with small batch sizes and introduces a Multi‑Scale Temporal Adapter to capture temporal interactions during transformer encoding. The framework also incorporates CSFormer, a CSLR‑guided spotting module that injects recognition‑derived gloss evidence into a boundary‑aware spotting network, enabling mutual benefit between recognition and spotting tasks.
arXiv Computation and Language
Sep 14
Investigating Temporal Motion Features for Pose-to-Text Indian Sign Language Translation
arXiv:2609.12993v1 Announce Type: cross Abstract: We investigate the effect of pretrained T5 model scale and explicit motion features on pose-to-text Indian Sign Language Translation (SLT) for the WS...
Hugging Face Trending Papers
Aug 26
SMART: MLLM-guided Temporal Alignment for Unifying Sign Language Recognition and Spotting
Continuous sign language recognition (CSLR) aims to recognize gloss sequences from unsegmented sign videos under weak sequence-level supervision. However, existing methods rely on sentence-level gloss...
arXiv AI
Aug 10
SignVerse-2M: A Two-Million-Clip Pose-Native Universe of 55+ Sign Languages
arXiv:2605. 01720v3 Announce Type: replace-cross Abstract: Existing large-scale sign language resources typically provide supervision only at the level of raw video-text alignment and are often produced in laboratory settings.
arXiv Computer Vision
Sep 3
SignMatch: Matching Dictionary Signs to Continuous Sign Language Video
SignMatch introduces a prototype‑structured embedding space that learns to match dictionary sign videos with continuous sign language footage based solely on visual similarity of handshape and motion. By mapping isolated dictionary exemplars into this space, the method enables direct, embedding‑based sign matching and can generalise to unseen signs using only dictionary examples. Experiments on ASL‑Citizen, ChaLearn OSLWL, and BOBSL CSLR2 benchmarks show strong cross‑dataset, cross‑task, and cross‑language performance, outperforming prior approaches on American, British, and Spanish sign languages without benchmark‑specific supervision.
arXiv AI
4d ago
Isharah-Selfie: Continuous Sign Language Recognition Dataset for One-handed Signing
Isharah‑Selfie is a large‑scale Arabic Sign Language dataset collected entirely with front‑facing smartphone cameras in unconstrained selfie conditions. It contains 8,997 video clips from 16 signers across 1,493 unique sentences, annotated with gloss sequences for continuous sign language recognition and Arabic sentence translations for sign language translation. The dataset focuses on realistic mobile communication, featuring one‑handed signing, close‑range viewpoints, partial body visibility, camera motion, unconstrained environments, and heterogeneous device resolutions, and introduces signer‑independent and unseen‑sentence evaluation protocols.
arXiv Computer Vision
Sep 21
SignGPT: Toward LLM-Mediated Sign Language Interaction through Gloss-Free Translation and Generation
SignGPT is a unified, pose‑based framework that performs gloss‑free sign language translation (SLT) and generation (SLG) by integrating part‑aware hierarchical representations of body, hand, and facial motion into a shared language model. It uses asymmetric multi‑token prediction and progressive training for bidirectional modeling, and is evaluated on How2Sign (ASL) and Phoenix‑2014T (DGS) with benchmark comparisons, qualitative analyses, and component ablations. An exploratory study with 12 Deaf ASL signers demonstrates a sign‑to‑sign response pipeline, suggesting that unified modeling can support sign language conversation (SLC).
Hugging Face Trending Papers
Jul 13
The In-Car Sign Language Corpus (ICSL): A Multi-Modal Resource for Constrained-Space Sign Language Recognition
This paper addresses the challenges of using sign language within shared mobility services, such as taxis, carpools, or ride-sharing platforms. The use of sign language recognition (SLR) in real-world, confined environments, specifically vehicle interiors remains largely unexplored.
arXiv AI
Jul 7
A Transformer-Based Contrastive Learning Approach for Few-Shot Sign Language Recognition
arXiv:2204. 02803v2 Announce Type: replace-cross Abstract: Sign language recognition from monocular video or 2D pose sequences is challenging, both because 3D information must be inferred from 2D observations and because the signal is inherently spatiotemporal.
arXiv Computer Vision
Sep 18
Attention-Steered Vision-Language Models for Sign Language Translation
The paper introduces AttnSign, a vision‑language model that improves sign language translation by steering spatial‑temporal attention. It first supervises attention on sign‑relevant regions such as faces and hands in each frame, then uses an RL‑based motion‑cadence method to focus on keyframes. Experiments on How2Sign and OpenASL show AttnSign consistently outperforms existing methods.