arXiv AI June 3, 2026 By Alexander Guha

Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models

arXiv:2606. 02765v1 Announce Type: cross Abstract: Model dimension ($d_{model}$) is a fundamental hyperparameter in transformer language models, yet its role in setting the geometric limits of feature representation remains under-explored.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Sep 24

Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces

arXiv:2609.27988v1 Announce Type: cross Abstract: Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direc...

By Andrew Bond, Ege Erdem \"Ozl\"u, Tuna \c{C}imen, Ilkin Umut Melanlioglu, Tolga Birdal, Erkut Erdem, Aykut Erdem

llms computer-vision fine-tuning efficiency

Hugging Face Trending Papers

Sep 23

Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces

Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direction is equally meaningful, but there is no reason...

llms computer-vision fine-tuning efficiency

arXiv Machine Learning

Jul 7

Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models

arXiv:2602. 24264v2 Announce Type: replace-cross Abstract: Compositional generalization, the ability to recognize familiar parts in novel contexts, is a defining property of intelligent systems.

By Arnas Uselis, Andrea Dittadi, Seong Joon Oh

rag

Hugging Face Trending Papers

Jun 28

Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings

The standard way to compare two text embeddings is cosine similarity. Scattered studies report that a different metric does better, but never pin down the geometric condition that decides when, or why.

llms rag fine-tuning

arXiv Machine Learning

Sep 22

Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities

arXiv:2609.24209v1 Announce Type: new Abstract: The Platonic Representation Hypothesis (PRH) claims that independently trained models converge on a shared statistical model of reality, yet recent wor...

By Chenming Shang, Yujin Tang, Jun Jie Ou Yang, Ruize Xu, Adam Breuer, Nikhil Singh

fine-tuning efficiency safety

arXiv Machine Learning

Jun 16

The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraints

arXiv:2606. 16028v1 Announce Type: new Abstract: Modern deep learning architectures are increasingly multi-task and multi-modal, using a pretrained foundation model combined with task-specific, fine-tuned models.

By Thomas Dittrich, Oliver Potocki, Philipp Grohs

fine-tuning multimodal

One email a morning, machine-written

Related stories

Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces

Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces

Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models

Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings

Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities

The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraints