SMART is a new framework that jointly tackles continuous sign language recognition (CSLR) and spotting by leveraging a multimodal large language model (MLLM) to generate motion descriptions as auxiliary semantic cues. It performs stable video‑text alignment with small batch sizes and introduces a Multi‑Scale Temporal Adapter to capture temporal interactions during transformer encoding. The framework also incorporates CSFormer, a CSLR‑guided spotting module that injects recognition‑derived gloss evidence into a boundary‑aware spotting network, enabling mutual benefit between recognition and spotting tasks.
By Eunjee Choi, JungHoon Sung, Seongwhan Cho, Chu Xin, Younggeun Choi
arXiv:2607. 03657v1 Announce Type: cross Abstract: Gloss-free Sign Language Translation (SLT) translates sign language videos into spoken-language sentences without gloss annotations, avoiding costly labeling but requiring fine-grained modeling of hands, body, and facial cues.
By Ahmed Abul Hasanaath, Bicheng Xu, Mir Rayat Imtiaz Hossain, Leonid Sigal, Hamzah Luqman
arXiv:2608. 06407v1 Announce Type: cross Abstract: Automated Sign Language Recognition for under-represented languages remains a largely unsolved problem.
By Lucia Yen Wanchi, Samuel Johnny, Victor Tolulope Olufemi, Emmanuel Aaron, Moise Busogi
arXiv:2606. 11925v1 Announce Type: cross Abstract: Sign language translation (SLT) converts sign language video into spoken language text and holds significant promise for improving accessibility and enabling communication between signing and non-signing communities.
By Zsolt Robotka, \'Ad\'am R\'ak, Jalal Al-Afandi, Andr\'as Horv\'ath, Gy\"orgy Cserey
Continuous sign language recognition (CSLR) is a key technology for accessibility, yet its development remains limited by the high cost of annotating continuous video streams. Active learning offers a...
RAIDAL is an active learning framework for continuous sign language recognition that leverages the CTC decoder’s alignment peaks to focus sample selection on gloss‑aligned regions, thereby avoiding temporal redundancy in weakly aligned videos. By restricting representation‑based scoring to these decoder‑aligned gloss areas, RAIDAL improves data efficiency across multiple datasets and architectures, especially in large‑vocabulary, budget‑limited scenarios. The method requires no extra labeling cost and its implementation is publicly available on GitHub.
By Rafael A. Diniz Augusto, Gabriel L. Oliveira, Erickson R. Nascimento
arXiv:2605. 01720v3 Announce Type: replace-cross Abstract: Existing large-scale sign language resources typically provide supervision only at the level of raw video-text alignment and are often produced in laboratory settings.
By Sen Fang, Hongbin Zhong, Yanxin Zhang, Dimitris N. Metaxas
arXiv:2609.05742v1 Announce Type: cross
Abstract: American Sign Language (ASL) generation remains challenging due to limited paired text-ASL motion data and the difficulty of learning motion represen...
By Hongyu Wu, Xu Wu, Tianhao Wu, Jiawei Yu, Phuc Nguyen, Jian Liu, Yi Wu
SignMatch introduces a prototype‑structured embedding space that learns to match dictionary sign videos with continuous sign language footage based solely on visual similarity of handshape and motion. By mapping isolated dictionary exemplars into this space, the method enables direct, embedding‑based sign matching and can generalise to unseen signs using only dictionary examples. Experiments on ASL‑Citizen, ChaLearn OSLWL, and BOBSL CSLR2 benchmarks show strong cross‑dataset, cross‑task, and cross‑language performance, outperforming prior approaches on American, British, and Spanish sign languages without benchmark‑specific supervision.
By Ryan Wong, Youngjoon Jang, Liliane Momeni, G\"ul Varol, Andrew Zisserman
arXiv:2204. 02803v2 Announce Type: replace-cross Abstract: Sign language recognition from monocular video or 2D pose sequences is challenging, both because 3D information must be inferred from 2D observations and because the signal is inherently spatiotemporal.
By Silvan Ferreira, Esdras Costa, Marcio Dahia, Jampierre Rocha
arXiv:2603. 29219v2 Announce Type: replace-cross Abstract: Sign language is the primary approach of communication for the Deaf and Hard-of-Hearing (DHH) community.
By Mohammad Amer Khalil, Raghad Nahas, Ahmad Nassar, Khloud Al Jallad
arXiv:2608. 09006v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable success across a wide range of tasks.
By Shiwei Gan, Xiao Liu, Yafeng Yin, Zhiwei Jiang, Bowen Guo, Lie Xie, Sanglu Lu, Hongkai Wen