arXiv Machine Learning By Juri Opitz, Andrianos Michail

Embedding Models Measure in Peculiar Ways

Read the original on arXiv Machine Learning →

The paper investigates whether embedding spaces capture objective physical measurements such as mass, distance, time, and volume. It finds that these embeddings only weakly model such measurements and instead exhibit peculiar patterns. Further analysis shows that superficial string similarity heavily influences the representation of physical measurements, and recalibrating similarity does not significantly improve alignment.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Jul 1

Symmetry in language statistics shapes the geometry of model representations

arXiv:2602. 15029v3 Announce Type: replace Abstract: The internal representations learned by language models consistently exhibit striking geometric structure: calendar months organize into a circle, historical years form a smooth one-dimensional manifold, and cities' latitudes and longitudes can be decoded using a linear probe.

By Dhruva Karkada, Daniel J. Korchinski, Andres Nava, Matthieu Wyart, Yasaman Bahri