arXiv Machine Learning
Predictive Geometry of Hidden Trajectories in Transformers
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Aug 14
Geometric and Behavioral Stratification in Transformer Residual Streams
arXiv:2608. 12447v1 Announce Type: new Abstract: Trained transformer models develop privileged bases: coordinate axes whose statistics differ from the rest of the residual stream.
arXiv Machine Learning
Sep 15
Disentangling Representation Evolution in Transformers through Directional Decomposition
arXiv:2609.15975v1 Announce Type: cross Abstract: Transformer representations evolve through learned additive transformations that either preserve their current direction or redirect it. We study thi...
arXiv Machine Learning
Sep 24
Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces
arXiv:2609.27988v1 Announce Type: cross Abstract: Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direc...
Hugging Face Trending Papers
Sep 23
Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces
Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direction is equally meaningful, but there is no reason...
arXiv AI
3d ago
Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regression
arXiv:2605. 08475v3 Announce Type: replace-cross Abstract: In this paper, we study in-context kernel ridge regression (KRR) with Gaussian kernels and show, both theoretically and empirically, that a standard softmax-attention transformer can approximate the KRR predictor during its forward pass.
arXiv Machine Learning
Jul 21
An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers
arXiv:2607. 17696v1 Announce Type: cross Abstract: We develop an adjoint-sensitivity framework for positional influence in causal residual Transformers and separate unconditional analytic results from conditional boundary-shape conclusions.