The paper argues that BLEU-4, the prevailing metric for sign language translation (SLT), may not accurately reflect sign language proficiency because SLT models can exploit spurious correlations and spoken-language priors. By evaluating six SLT models on Phoenix-2014T and CSL-Daily, the authors show that higher BLEU-4 scores do not necessarily indicate better spatio-temporal understanding. They propose a new open-weight LLM QA protocol inspired by language-learning assessment, which better preserves salient content, aligns more closely with human rankings, and reveals differences between gloss-free and gloss-supervised systems that BLEU-4 obscures.
arXiv:2609.07965v1 Announce Type: cross Abstract: Sign Language Translation has advanced with deep learning, yet evaluations remain largely signer-dependent, with overlapping signers across train/dev...
SignRAG is a unified framework for gloss‑free sign language translation that combines hierarchical pretraining, target‑domain retrieval augmentation, and retrieval‑aware reinforcement fine‑tuning. It first learns linguistically grounded sign representations, aligns a sign encoder with a decoder‑only large language model, and then uses a retrieval gallery to provide instance‑specific translation cues. Retrieval Utility‑Guided Reinforcement Fine‑Tuning (RUG‑RFT) balances translation quality and retrieval usefulness, leading to state‑of‑the‑art results, including the first gloss‑free approach to outperform gloss‑supervised methods on CSL‑Daily.
arXiv:2605.20588v2 Announce Type: replace Abstract: Sign language translation has made substantial progress between sign and spoken languages, while translation across sign languages remains less exp...
SignSeek is a new method for learning transferable sign representations that enables efficient retrieval of signs from dictionaries using only a query video. It employs contrastive learning with saliency‑guided articulator masking, aligning same‑gloss signs across signers while focusing on the single most critical articulator per sign. Trained on 266K samples from multiple sign languages, SignSeek achieves state‑of‑the‑art cross‑corpus retrieval performance and zero‑shot generalisation to unseen British Sign Language, also improving isolated sign recognition and subtitle alignment.
arXiv:2605. 31393v2 Announce Type: replace-cross Abstract: Sign language translation (SLT) remains constrained by the limited availability of paired sign-video/text corpora and by the heavy-tailed vocabularies typical of real-world datasets.
arXiv:2609.25862v1 Announce Type: new Abstract: We present the first experiments on isolated sign language recognition (ISLR) for Icelandic Sign Language (\'ITM). We use \'ITM SignWiki, a dataset der...
SignGPT is a unified, pose‑based framework that performs gloss‑free sign language translation (SLT) and generation (SLG) by integrating part‑aware hierarchical representations of body, hand, and facial motion into a shared language model. It uses asymmetric multi‑token prediction and progressive training for bidirectional modeling, and is evaluated on How2Sign (ASL) and Phoenix‑2014T (DGS) with benchmark comparisons, qualitative analyses, and component ablations. An exploratory study with 12 Deaf ASL signers demonstrates a sign‑to‑sign response pipeline, suggesting that unified modeling can support sign language conversation (SLC).
arXiv:2606. 19352v1 Announce Type: cross Abstract: Sign languages are expressive visual languages used by Deaf and Hard-of-Hearing (DHH) communities.
arXiv:2609.12993v1 Announce Type: cross Abstract: We investigate the effect of pretrained T5 model scale and explicit motion features on pose-to-text Indian Sign Language Translation (SLT) for the WS...
SignBind-LLM introduces a modular framework for sign language translation that separates continuous signing, fingerspelling, and lipreading into dedicated expert streams. Each expert is pre‑trained independently on about two million pseudo‑gloss sequences, eliminating the need for manual gloss annotation. A lightweight transformer fuses the expert outputs, and a pre‑trained language model converts the fused pseudo‑glosses into fluent English, achieving state‑of‑the‑art performance on multiple benchmarks with lower training cost.
The paper introduces BdSLIG, the first Bengali Sign Language Instruction Generation dataset, aimed at evaluating Vision Language Models on under-resourced SLIG tasks and long-tail visual concepts. It proposes Sign Parameter-Infused (SPI) prompting, which embeds standard sign parameters such as hand shape, motion, and orientation into textual prompts to improve zero-shot performance and produce more structured, reproducible instructions. The work seeks to promote inclusivity and advance sign language learning systems for under-resourced communities.
The paper introduces a reverse sign‑language dictionary that recognizes signs from continuous signing without relying on gloss labels. It does this by first captioning a sign‑level video clip into a free‑form procedural description using an open‑weight vision‑language model, then retrieving the closest entry from a multilingual sentence encoder’s vocabulary of target descriptions. Experiments on a Japanese Sign Language dialogue corpus show that fine‑tuning the captioner boosts seen‑class retrieval from 4.5 % to 49 % and improves unseen‑class retrieval from 11.5 % to 21 %, approaching the performance of a standard closed‑set classifier while enabling open‑vocabulary recognition.