arXiv AI
Jul 9
Riemannian Geometry for Pre-trained Language Model Embeddings
arXiv:2607. 07047v1 Announce Type: cross Abstract: Understanding the geometric structure of pre-trained language model embeddings matters for interpretability and safety.
An empirical investigation into the properties of standard word embeddings
arXiv:2607. 23675v1 Announce Type: cross Abstract: The embedding of word sequences into continuous vector spaces has been one of the most important developments in Natural Language Processing in the recent past.
Related stories
arXiv AI
Jun 18
Morpheus: A Morphology-Aware Neural Tokenizer and Word Embedder for Turkish
arXiv:2606. 18717v1 Announce Type: cross Abstract: Turkish is agglutinative: meaning is carried by morphemes, yet the subword tokenizers that drive modern language models split words by corpus statistics, fragmenting semantically loaded suffixes and -- in the case of WordPiece and rule-based analyzers -- failing to decode their output back to the original text.
arXiv Machine Learning
Jul 28
BHARATI: Morphology-Aware Tokenizers for Classical Indian Languages with Subword Fertility Analysis
arXiv:2607. 23319v1 Announce Type: cross Abstract: Standard subword tokenization algorithms such as Byte-Pair Encoding (BPE) and SentencePiece are trained predominantly on modern language corpora and produce inefficient segmentations when applied to classical Indian languages.
arXiv Machine Learning
Jul 8
Where to cut, how deep: BPE and Unigram-LM on chemistry SMILES
arXiv:2607. 05691v1 Announce Type: cross Abstract: Every chemical language model reading SMILES begins with a tokenizer, yet the field has inherited byte-pair encoding (BPE) from natural language with little scrutiny.
Hugging Face Trending Papers
Jun 28
Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings
The standard way to compare two text embeddings is cosine similarity. Scattered studies report that a different metric does better, but never pin down the geometric condition that decides when, or why.
arXiv Machine Learning
Jul 7
The Truncation Blind Spot: How Decoding Strategies Systematically Exclude Human-Like Token Choices
arXiv:2603. 18482v2 Announce Type: replace-cross Abstract: Standard decoding strategies for text generation, including top-$k$, nucleus sampling, and contrastive search, select tokens based on likelihood, restricting outputs to high-probability regions.
arXiv Machine Learning
Aug 4
Automatic Annotation of Ancient Greek Vowel Length
arXiv:2608. 01935v1 Announce Type: cross Abstract: Prior work in Ancient Greek NLP relies on corpora that do not disambiguate the phonemic vowel length of alpha, iota, and ypsilon, together known as the dichrona.
arXiv Machine Learning
Jul 21
Perturbation is All You Need for Extrapolating Language Models
arXiv:2605. 04344v2 Announce Type: replace-cross Abstract: This paper develops a statistical theory of extrapolation for large language models, by reinterpreting them through pre-post-additive noise models.
arXiv AI
Jun 17
findsylls: A Language-Agnostic Toolkit for Syllable-Level Speech Tokenization and Embedding
arXiv:2603. 26292v2 Announce Type: replace-cross Abstract: Syllable-level units offer compact and linguistically meaningful representations for spoken language modeling and unsupervised word discovery, but research on syllabification remains fragmented across disparate implementations, datasets, and evaluation protocols.
arXiv AI
Aug 3
Guarantees on Dynamical System Distinguishability for LLM Token Generation
arXiv:2607. 28667v1 Announce Type: cross Abstract: Recent work has shown that classifying large language models (LLMs)' responses can be distinguished by modeling token embeddings as trajectories of a black-box dynamical system (DS) and comparing prediction residuals of two DSs.
Towards Data Science
May 30
Embeddings Aren’t Magic: The Predictable Failure Modes of RAG Retrieval
Enterprise Document Intelligence [Vol. 1 #2] Why the same vector search that handles synonyms and paraphrase silently fails on negation, exact identifiers, and your company’s acronyms, and what to use when it does.
arXiv AI
Jun 9
Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics
arXiv:2606. 08417v1 Announce Type: cross Abstract: Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling.