← Back to all news
arXiv Computation and Language By Tim Wientzek

Self-Supervised Lexical Representation Learning for Fast, Large-Scale Phylogenetic Inference

Read the original on arXiv Computation and Language →

The paper presents a fully self‑supervised contrastive learning framework that learns lexical representations from raw IPA‑transcribed wordlists without any cognacy annotations or expert input. Using a dual contrastive objective—word‑level and language‑level losses—the model produces word representations that enable fast computation of pairwise language distances and the inference of a global phylogenetic tree for 3,399 language varieties. The resulting tree achieves a generalized quartet distance competitive with baselines while requiring only minutes of computation on a standard notebook GPU, and the representations also capture diachronic concept stability.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Machine Learning
Oct 9

RAGenome: Scaling Retrieval-Based Genomic Language Models to Long Contexts

RAGenome is a retrieval‑based genomic language model that extends pretraining to contexts 100 times longer than existing MSA‑based models, enabling it to capture both cross‑species evolutionary relationships and within‑species long‑range interactions. Trained on whole‑genome alignments from 100 vertebrates, it improves gene‑finding performance from 0.45 to 0.60 while staying competitive on evolutionary tasks such as prioritizing pathogenic variants. The model offers competitive performance at a fraction of the training cost, unifying evolutionary modeling and long‑range capabilities in a single scalable framework.

By Frederikke Isa Marin, Panagiotis Antoniadis, Dionysia Danai Brilli, Andreas Bjerregaard, Rachael DeVries, Yan Li, Ole Winther, Wouter Boomsma
llms
More like this →
arXiv Computation and Language
Sep 21

Beyond Atomic Tokens: Factorizing Syllables for Language Model Pretraining

arXiv:2609. 21362v1 Announce Type: new Abstract: Conventional tokenizers represent text as characters or statistically derived subwords, overlooking the internal phonological structure of syllables and often requiring large vocabularies.

By Nghia Hieu Nguyen, Thai Bao Huynh, Binh-An Dinh-Le, Phu Gia Hoang, Dat Tien Nguyen, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen
llmsragnlp
More like this →
arXiv Computation and Language
Aug 31

Phoneme- and Word-Level Metrics Using Self-Supervised Speech Representations for Forced Alignment Evaluation

The paper introduces two new corpus‑level, reference‑free metrics—Phoneme‑Cluster Mutual Information (PCMI) and Word Acoustic Consistency Score (WACS)—that use self‑supervised speech representations to evaluate forced alignment quality. PCMI quantifies how well aligned phoneme labels agree with clusters derived from SSL representations, while WACS assesses consistency across repeated word realizations via dynamic time warping of word representation sequences. Experiments on 85 languages from FLEURS and 45 languages in DoReCo show that both metrics degrade predictably under alignment perturbations, effectively distinguish high‑ from low‑quality alignments, and correlate strongly with traditional timestamp‑based measures, enabling scalable, multilingual alignment evaluation without manual annotations.

By V. S. D. S. Mahesh Akavarapu, Michael Daniel, Gerhard J\"ager
safety
More like this →