The paper proves the Platonic Representation Hypothesis by showing that Joint-Embedding Predictive Architectures (JEPAs) trained on different modalities converge to the same representation up to a rotation when their positive pairs are good views of the same scene. It demonstrates that next‑token prediction is a JEPA with bad views, explaining why language models only align with vision through large scale. Experiments on millions of captions and web images confirm a scaling law of alignment, revealing that good views require far fewer samples than next‑token prediction and that a 109M‑parameter text JEPA outperforms larger language models in aligning with image encoders, even enabling a simple rotation to produce an image captioner with high COCO CIDEr scores.
arXiv:2604. 18572v2 Announce Type: replace-cross Abstract: The Platonic Representation Hypothesis suggests that neural networks trained on different modalities (e.
arXiv:2609.24209v1 Announce Type: new Abstract: The Platonic Representation Hypothesis (PRH) claims that independently trained models converge on a shared statistical model of reality, yet recent wor...
The Platonic Representation Hypothesis (PRH) claims that independently trained models converge on a shared statistical model of reality, yet recent work finds only weak pointwise similarity between mo...
arXiv:2509. 07295v4 Announce Type: replace-cross Abstract: Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture.
The paper proposes that independently trained multimodal models often share a common geometric structure, allowing cross‑modal alignment without paired data. Using a Wasserstein Procrustes approach with a coarse geometric initialization, the authors align two disjoint embedding sets by estimating a single orthogonal map, achieving consistent alignment across datasets and modalities. While paired examples can further improve performance, the method already outperforms existing techniques in the few‑pair regime and enables text‑to‑image generation without paired data.
The paper introduces the Lattice Representation Hypothesis (LRH) to explain why different alignment criteria—such as neighbor overlap, spanning trees, and distances—yield varying views on model representation convergence. It shows that the Platonic Representation Hypothesis (PRH) reflects agreement at the top of a lattice, while evidence comes from criteria near the bottom, and that the Aristotelian Representation Hypothesis (ARH) limits convergence to local neighborhoods due to calibration effects. Experiments on numerous pretrained vision and language models, along with theoretical analyses of linear autoencoders and contrastive Gibbs models, support the LRH’s claim that training procedures and objectives determine which criteria are reliably met across runs.
DEPICT is a new training‑free metric for evaluating text‑to‑image alignment. It replaces fixed reference answers with an agreement rule that compares image‑based and caption‑only responses, weighting questions by how decisively the caption determines them. By merging this agreement score with a holistic score, DEPICT improves negation accuracy dramatically and outperforms existing training‑free metrics while matching or exceeding fine‑tuned evaluators on several benchmarks.
arXiv:2610.33798v1 Announce Type: cross Abstract: An image is rarely described only once. Retrieval benchmarks and many collections attach several captions to each picture, yet dual encoders still ju...
TokenSwap introduces a method to benchmark and reduce the modality gap in multimodal large language models (MLLMs) by interleaving semantically aligned images with text tokens. By transforming text-only benchmarks like MMLU into image‑interleaved versions (TokenSwap‑Bench), the authors reveal a pervasive performance drop—averaging 19.6%—across 42 MLLMs, with reasoning models showing smaller gaps. Training with TokenSwap mitigates this gap while maintaining strong text‑only and vision‑language performance.
arXiv:2609.07937v1 Announce Type: cross Abstract: Structured visual reasoning, such as image puzzles, demands fine-grained visual perception, an ability current Vision Language Models (VLMs) lack. VL...
arXiv:2609.12552v1 Announce Type: new Abstract: Open-vocabulary detection accepts any class list at inference, and promptable segmentation returns regions without class names: the taxonomy has left t...
arXiv:2606. 05173v1 Announce Type: cross Abstract: Masked language modelling (MLM) has been the dominant pre-training objective for text encoders since BERT, yet it encourages representations that are strongly anchored to surface-form token identity rather than deeper semantic structure.