The paper presents a fully self‑supervised contrastive learning framework that learns lexical representations from raw IPA‑transcribed wordlists without any cognacy annotations or expert input. Using a dual contrastive objective—word‑level and language‑level losses—the model produces word representations that enable fast computation of pairwise language distances and the inference of a global phylogenetic tree for 3,399 language varieties. The resulting tree achieves a generalized quartet distance competitive with baselines while requiring only minutes of computation on a standard notebook GPU, and the representations also capture diachronic concept stability.
arXiv:2610.31894v1 Announce Type: new Abstract: Recent Sparse Autoencoders (SAEs) decompose the representations of large language models (LLMs) into interpretable features, enabling precise readout a...
RAGenome is a retrieval‑based genomic language model that extends pretraining to contexts 100 times longer than existing MSA‑based models, enabling it to capture both cross‑species evolutionary relationships and within‑species long‑range interactions. Trained on whole‑genome alignments from 100 vertebrates, it improves gene‑finding performance from 0.45 to 0.60 while staying competitive on evolutionary tasks such as prioritizing pathogenic variants. The model offers competitive performance at a fraction of the training cost, unifying evolutionary modeling and long‑range capabilities in a single scalable framework.
arXiv:2607. 22699v1 Announce Type: new Abstract: As Large Language Models (LLMs) grow more capable across diverse tasks, their (in)ability to generalize remains difficult to quantify and poorly understood beyond limited domains.
arXiv:2609. 21362v1 Announce Type: new Abstract: Conventional tokenizers represent text as characters or statistically derived subwords, overlooking the internal phonological structure of syllables and often requiring large vocabularies.
arXiv:2610.08851v1 Announce Type: new Abstract: Quantifying language distance among closely related languages remains a core challenge in quantitative linguistics. Our previous work [1] introduced Qu...
The paper introduces two new corpus‑level, reference‑free metrics—Phoneme‑Cluster Mutual Information (PCMI) and Word Acoustic Consistency Score (WACS)—that use self‑supervised speech representations to evaluate forced alignment quality. PCMI quantifies how well aligned phoneme labels agree with clusters derived from SSL representations, while WACS assesses consistency across repeated word realizations via dynamic time warping of word representation sequences. Experiments on 85 languages from FLEURS and 45 languages in DoReCo show that both metrics degrade predictably under alignment perturbations, effectively distinguish high‑ from low‑quality alignments, and correlate strongly with traditional timestamp‑based measures, enabling scalable, multilingual alignment evaluation without manual annotations.
CWoMP (Contrastive Word‑Morpheme Pretraining) is a new approach for generating interlinear glossed text that treats morphemes as atomic form‑meaning units with learned representations. It uses a contrastively trained encoder to align words in context with their constituent morphemes in a shared embedding space, and an autoregressive decoder that retrieves morpheme sequences from a mutable lexicon of these embeddings. The method yields interpretable predictions grounded in lexicon entries and allows users to improve results at inference time by expanding the lexicon without retraining, achieving superior performance and efficiency on diverse low‑resource languages, especially in extremely low‑resource settings.
arXiv:2608. 01935v1 Announce Type: cross Abstract: Prior work in Ancient Greek NLP relies on corpora that do not disambiguate the phonemic vowel length of alpha, iota, and ypsilon, together known as the dichrona.
arXiv:2610.31723v1 Announce Type: new Abstract: This article introduces Latin Modern BERT (LAMB), an encoder for the Latin language based on the ModernBERT architecture, pretrained in two phases on m...
The paper investigates how phylogenetic leakage can inflate performance estimates for protein classifiers, particularly in venom ion‑channel target prediction. By treating cross‑species prediction as a structured distribution shift, the authors compare five structural inductive biases using a multimodal graph neural network that fuses ESM‑2 embeddings with a disulfide‑aware attention mechanism. Persistent‑homology representations, which capture backbone geometry through topological descriptors, outperform other biases under a leave‑one‑taxon‑out protocol, achieving 63.4% accuracy versus 54‑55% for other geometric priors and 30% for a sequence‑only baseline.
PunGraph is a retrieval‑enhanced knowledge‑graph framework designed to improve pun understanding by combining phonetic and semantic information. It builds a phonetic‑semantic lexical graph using the Unisyn phonetic dictionary, IPA and G2P representations, and WordNet definitions, and uses this graph to retrieve candidate words or senses that constrain large language model reasoning. The authors also introduce WebPun, a new dataset of 5,730 annotated heterographic and homographic puns, and demonstrate that PunGraph consistently boosts the performance of small‑scale LLMs on SemEval‑2017 and WebPun, achieving results competitive with strong proprietary models.
TatBLiMP is the first benchmark of linguistic minimal pairs for the Tatar language, covering 16 morphosyntactic phenomena across 1,248 sentence pairs that differ by a single morpheme. Each pair contains one grammatical and one ungrammatical sentence, with the ungrammatical version generated by a deterministic perturbation and ratified by a native speaker. The benchmark evaluates models by comparing their assigned probabilities, allowing assessment without text generation or parsing, and tracks performance across from-scratch, cross‑lingual, and multilingual large language models.