arXiv Machine Learning September 21, 2026 By Vasudevan Nedumpozhimana, Fathima Thekkekara, John Kelleher

Analysing the Linearity of Linguistic Relations in Language Model Embedding Spaces

Read the original on arXiv Machine Learning →

The paper introduces a framework to assess how linearly encoded linguistic relations are within language model embeddings. Using a constrained linear approximation on related and unrelated word pairs, the authors evaluate an extended BATS dataset across inflectional, derivational, lexicographic, and encyclopedic relations in GloVe, RoBERTa, and ModernBERT. Results show near-perfect linear encoding for inflectional and derivational relations, higher errors for lexicographic and encyclopedic relations—especially in one-to-many and many-to-many cases—and that RoBERTa and ModernBERT encode relations more linearly than GloVe.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computation and Language

Aug 31

Representation of syntax in LLMs through the lens of linear distance and similarity-aware entropy

arXiv:2608.27813v1 Announce Type: new Abstract: Structural probes were introduced by Hewitt and Manning to reconstruct syntactic trees from a neural language model's latent representations. They are...

By Juan Pablo Vigneaux, Mary Kennedy, Khalil Iskarous, Robert Frank, Matilde Marcolli

llms rag

arXiv Computation and Language

6d ago

Inductive Learning of Concept Embeddings from Cross-Linguistic Colexification Patterns with Textual Attributes

The paper introduces an inductive method for learning concept embeddings from cross‑linguistic colexification networks combined with short textual descriptions. Unlike previous transductive approaches, this framework can embed unseen concepts using only a brief definition. The resulting embeddings correlate with cross‑linguistic similarity measures and generalize well to new concepts.

By Arne Rubehn

rag

arXiv AI

Aug 5

Reversing Arrows in Large Language Models

arXiv:2608. 03512v1 Announce Type: new Abstract: Large language models (LLMs) have achieved strong performance on text-to-knowledge graph generation and related tasks.

By Sefika Efeoglu, Adrian Paschke

llms benchmarks

arXiv AI

Oct 7

World Properties without World Models: Distributional Associations and the Interpretation of Decoding Results from Language Models

arXiv:2603.04317v2 Announce Type: replace-cross Abstract: A growing literature shows that variables can be linearly decoded from the activations of large language models (LLMs). These range from prop...

By Elan Barenholtz

llms rag

arXiv Computation and Language

Sep 22

The Harder Text Embedding Benchmark (HTEB): Beyond One-dimensional Static Robustness

arXiv:2605.28190v2 Announce Type: replace Abstract: Embedding benchmarks like MTEB report a single score per model, implicitly treating robustness as a static, scalar property. We argue that embeddin...

By Manuel Frank, Haithem Afli

llms rag benchmarks

arXiv Computation and Language

Sep 2

ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairs

arXiv:2607.00171v2 Announce Type: replace Abstract: Text embeddings are standard for semantic similarity tasks, yet their evaluation remains an open challenge. Current benchmarks are static, cover on...

By Andrianos Michail, Stylianos Psychias, Michelle Wastl, Simon Clematide, Rico Sennrich, Juri Opitz

rag nlp benchmarks

One email a morning, machine-written

Related stories

Representation of syntax in LLMs through the lens of linear distance and similarity-aware entropy

Inductive Learning of Concept Embeddings from Cross-Linguistic Colexification Patterns with Textual Attributes

Reversing Arrows in Large Language Models

World Properties without World Models: Distributional Associations and the Interpretation of Decoding Results from Language Models

The Harder Text Embedding Benchmark (HTEB): Beyond One-dimensional Static Robustness

ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairs