arXiv AI
Jul 7
Multi-Way Representation Alignment
arXiv:2602. 06205v2 Announce Type: replace-cross Abstract: The Platonic Representation Hypothesis suggests that independently trained neural networks converge to increasingly similar latent spaces.
Native Geometry vs. Linear Recoverability: How Alignment Changes Platonic Scaling
Related stories
arXiv Machine Learning
Sep 24
What Converges in the Platonic Representation Hypothesis? Structure over Geometry
The paper investigates the Platonic Representation Hypothesis, which posits that more capable models converge toward shared representations. By distinguishing relational structure (which samples are related) from metric geometry (quantitative relations like distances), the authors develop a controlled $2 imes2$ framework to evaluate both aspects at local and global scales. Their findings show that relational structure consistently converges across vision‑language and video‑text models, while metric geometry converges much more weakly, a pattern that persists even when using a Riemannian metric approximation.
arXiv AI
Jun 3
Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale
arXiv:2604. 18572v2 Announce Type: replace-cross Abstract: The Platonic Representation Hypothesis suggests that neural networks trained on different modalities (e.
arXiv Machine Learning
Jul 8
Geometric Stability: The Missing Axis of Representations
arXiv:2601. 09173v5 Announce Type: replace Abstract: Representational similarity analysis and related methods compare the internal geometries of neural networks, but they measure only alignment between spaces, leaving a blind spot -- whether a representation's structure is reliably recoverable, not merely similar.
arXiv Computer Vision
Oct 1
Spherical Interpolation for Backward-Compatible Multimodal Representations
arXiv:2609.39836v1 Announce Type: new Abstract: Contrastive vision-language models map visual and textual representations into a shared normalized embedding space, making cosine similarity the natura...
arXiv Machine Learning
Jun 16
Size Doesn't Matter: Cosine-Scored Sparse Autoencoders
arXiv:2606. 15054v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) detect features via inner product, so a feature's activation scales with both its directional alignment and the input's norm.
arXiv Machine Learning
Sep 1
Unsupervised Latent Space Alignment with Hyperspherical Geodesic Matching
arXiv:2608.28840v1 Announce Type: new Abstract: Independently trained neural networks tend to encode the same data with similar latent geometries. These latent geometries are not directly compatible,...
arXiv Machine Learning
Jun 10
In Defense of Cosine Similarity: Normalization Eliminates the Gauge Freedom
arXiv:2602. 19393v2 Announce Type: replace Abstract: Steck, Ekanadham, and Kallus [arXiv:2403.
arXiv Machine Learning
Oct 3
Beyond Linear Concepts: Discovering and Aligning Non-Linear Concept Manifolds in Large Language Models
The paper extends mechanistic interpretability of large language models by modeling concepts as low‑dimensional non‑linear manifolds rather than linear subspaces. It introduces a concept‑based alignment (CBA) score to compare these manifolds across layers and models, revealing block structures in intermediate layers, a shift from syntax‑dominated to mixed syntactic‑semantic concepts, and training‑dependent multilingual sharing. The study also shows that alignment patterns differ across model families and training stages, with adjacent stages aligning more closely than distant ones.
Hugging Face Trending Papers
Sep 23
Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces
Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direction is equally meaningful, but there is no reason...
arXiv Machine Learning
Oct 8
Shared Geometry As A Rosetta Stone: Cross-Modal Alignment Without Paired Data
The paper proposes that independently trained multimodal models often share a common geometric structure, allowing cross‑modal alignment without paired data. Using a Wasserstein Procrustes approach with a coarse geometric initialization, the authors align two disjoint embedding sets by estimating a single orthogonal map, achieving consistent alignment across datasets and modalities. While paired examples can further improve performance, the method already outperforms existing techniques in the few‑pair regime and enables text‑to‑image generation without paired data.
arXiv Machine Learning
Jul 24
Semantics at an Angle: When Cosine Similarity Works Until It Doesn't
arXiv:2504. 16318v3 Announce Type: replace Abstract: Cosine similarity is a standard comparison rule for learned representations in information retrieval, natural language processing, computer vision, and multimodal learning.