Hugging Face Trending Papers

Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities

arXiv AI
Oct 31

Proving and Improving the Platonic Representation Hypothesis with JEPAs

The paper proves the Platonic Representation Hypothesis by showing that Joint-Embedding Predictive Architectures (JEPAs) trained on different modalities converge to the same representation up to a rotation when their positive pairs are good views of the same scene. It demonstrates that next‑token prediction is a JEPA with bad views, explaining why language models only align with vision through large scale. Experiments on millions of captions and web images confirm a scaling law of alignment, revealing that good views require far fewer samples than next‑token prediction and that a 109M‑parameter text JEPA outperforms larger language models in aligning with image encoders, even enabling a simple rotation to produce an image captioner with high COCO CIDEr scores.

By Randall Balestriero, Yann LeCun
arXiv Machine Learning
Oct 15

From Plato to Aristotle: Toward a General Theory of Representation Hypotheses

The paper introduces the Lattice Representation Hypothesis (LRH) to explain why different alignment criteria—such as neighbor overlap, spanning trees, and distances—yield varying views on model representation convergence. It shows that the Platonic Representation Hypothesis (PRH) reflects agreement at the top of a lattice, while evidence comes from criteria near the bottom, and that the Aristotelian Representation Hypothesis (ARH) limits convergence to local neighborhoods due to calibration effects. Experiments on numerous pretrained vision and language models, along with theoretical analyses of linear autoencoders and contrastive Gibbs models, support the LRH’s claim that training procedures and objectives determine which criteria are reliably met across runs.

By Dai Shi, Andi Han, Xiaoyu Li, Luke Thompson, Jos\'e Miguel Hern\'andez-Lobato
arXiv Computer Vision
Oct 27

Do vision models know that three is three?

arXiv:2610.35857v1 Announce Type: new Abstract: Do vision models encode the transition $2{\to}3$ the same way whether it appears in handwritten digits or in hand gestures, and do they "understand" th...

By Ankit Sonthalia, Arnas Uselis, Seong Joon Oh
arXiv Machine Learning
Oct 8

Shared Geometry As A Rosetta Stone: Cross-Modal Alignment Without Paired Data

The paper proposes that independently trained multimodal models often share a common geometric structure, allowing cross‑modal alignment without paired data. Using a Wasserstein Procrustes approach with a coarse geometric initialization, the authors align two disjoint embedding sets by estimating a single orthogonal map, achieving consistent alignment across datasets and modalities. While paired examples can further improve performance, the method already outperforms existing techniques in the few‑pair regime and enables text‑to‑image generation without paired data.

By Dominik Schnaus, Thomas Dag\`es, Daniel Cremers, Xi Wang, Phillip Isola