← Back to all news
arXiv Machine Learning

Exact semantic readout from compressed vector representations

The paper investigates when compressed vector representations can provide exact linear or affine readouts for a finite lexicon’s truth conditions, establishing a necessary and sufficient row‑space condition. It shows that the augmented truth matrix’s rank determines the minimal dimension needed for exact linear (rank r) and affine (rank r − 1) readouts, and that exact readouts preserve Boolean connectives. Experiments on GloVe and word2vec embeddings reveal that while many predicates are linearly separable, none achieves exact affine recovery from pretrained embeddings, yet supervised transductive training can attain exact affine recovery at dimensions meeting the theoretical bound, preserving most of the original variance. "whyItMatters":"The results provide a precise mathematical criterion for when vector embeddings can faithfully encode logical truth conditions, informing both theoretical understanding and practical training of language models."

Related stories

arXiv Machine Learning
Sep 3

The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models

The paper investigates how truth representations in small language models are structured. Using a training‑free axis derived from the dominant singular vector of hidden‑state differences between true and false minimal pairs, the authors evaluate 14 models across six architectural families, including Mixture‑of‑Experts. The study examines whether a single direction captures truth, which components contribute, and how this applies to categories with computed truth values.

By Francesco Karim Vicidomini
llms
More like this →
arXiv Machine Learning
Sep 2

Hidden relationships in a document-derived property graph: top-k chunk embeddings and inverse-distance weighting over a dynamically evolving ontology

The paper introduces a second-pass method for uncovering hidden relationships in knowledge graphs extracted from text, without altering the original facts. By chunking documents and embedding each chunk once, the approach uses top‑k nearest‑neighbor queries and Shepard inverse‑distance weighting to score candidate node pairs, avoiding threshold issues inherent in cosine scoring. The technique is order‑independent, scalable, and has been implemented across multiple graph databases, demonstrating high edge fidelity with lower‑dimensional embeddings and a 25× speedup in top‑k computation.

By Bilge Kaan Karamete, Hunter Casten
llmsrag
More like this →