arXiv Machine Learning
Invariant Atoms: Sparse Coordinates of Local Semantic Geometry in Language Model Representations
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
5d ago
Fisher-IRG: Fisher-Induced Local Invariant Representation Geometry across Language and Vision Models
Semantic-preserving transformations can induce substantial motion in learned representations, while small changes may strongly affect model predictions, raising a basic question: what local metric bes...
arXiv Machine Learning
Jul 30
Using large language models to probe the limits of atom-centered structural descriptors
arXiv:2607. 26984v1 Announce Type: cross Abstract: Mapping an atomic structure to a compact set of geometric descriptors is an essential step in any machine-learning application to atomic-scale modeling.
arXiv Machine Learning
4d ago
Fisher-IRG: Fisher-Induced Local Invariant Representation Geometry across Language and Vision Models
arXiv:2609.36458v1 Announce Type: new Abstract: Semantic-preserving transformations can induce substantial motion in learned representations, while small changes may strongly affect model predictions...
arXiv AI
Jun 11
Augmenting Molecular Language Models with Local $n$-gram Memory
arXiv:2606. 12113v1 Announce Type: cross Abstract: Transformer-based language models for SMILES strings suffer from a locality gap: standard character-level tokenization fragments chemically meaningful motifs, forcing models to repeatedly learn local syntax at the expense of long-range dependencies.
arXiv Machine Learning
Jul 1
Symmetry in language statistics shapes the geometry of model representations
arXiv:2602. 15029v3 Announce Type: replace Abstract: The internal representations learned by language models consistently exhibit striking geometric structure: calendar months organize into a circle, historical years form a smooth one-dimensional manifold, and cities' latitudes and longitudes can be decoded using a linear probe.
arXiv Machine Learning
Sep 11
The information geometry of large language models is shared, learned, and controllable
The paper investigates how large language models (LLMs) share a common Fisher‑Rao geometry in their next‑token probability distributions, revealing that behaviour largely determines this geometry while activation geometry depends on coordinate choices. Across transformer, state‑space, and recurrent architectures, output geometries align more closely than activation geometries, and this shared structure facilitates semantic‑category transfer and improves agreement with human word choices as models scale and train. The study further demonstrates that geometry can guide minimum‑disturbance interventions, enabling reusable control that preserves behaviour better than Euclidean methods and enhances steering, editing, attribution, dictionary learning, and fine‑tuning.