arXiv:2606. 12146v1 Announce Type: cross Abstract: Rotary Position Embedding (RoPE) is widely adopted in Transformer models, yet its extension to high-dimensional domains lacks a unified theoretical formulation.
arXiv:2406. 07049v3 Announce Type: replace-cross Abstract: Understanding spatial relationships across all dimensions is fundamental for intelligent systems.
arXiv:2605. 29987v2 Announce Type: replace Abstract: Although multi-scales representation learning enables elastic-dimension embeddings, nested subspaces often suffer from dimensional redundancy and spectral collapse.
Despite the widespread adoption of Vision Transformers (ViTs) and their success across numerous computer vision applications, the fundamental understanding of their dimensional and representational geometry remains relatively underexplored. To address this gap, we introduce Transformer Geometry Observatory (TGO), a systematic framework of experiments and analysis pipelines designed to investigate the representational geometry and dynamics of Vision Transformers.
arXiv:2602. 10099v2 Announce Type: replace Abstract: Leveraging representation encoders for generative modeling offers a path for efficient, high-fidelity synthesis.
arXiv:2606. 03260v1 Announce Type: cross Abstract: Deep learning surrogates for 3D Partial Differential Equations (PDEs) often fail to generalize across geometric transformations because they depend heavily on specific coordinate systems.
arXiv:2606. 31061v1 Announce Type: cross Abstract: Tensor Train (TT) decomposition is a powerful technique for analyzing high-dimensional data.
arXiv:2606. 19249v1 Announce Type: cross Abstract: Despite the widespread adoption of Vision Transformers (ViTs) and their success across numerous computer vision applications, the fundamental understanding of their dimensional and representational geometry remains relatively underexplored.
arXiv:2601. 19179v2 Announce Type: replace Abstract: Autoencoders have long been considered a nonlinear extension of Principal Component Analysis (PCA).
arXiv:2607. 08987v1 Announce Type: new Abstract: Spectral embedding methods are widely used for dimensionality reduction and clustering of high-dimensional datasets with intrinsic low-dimensional structures.
arXiv:2505. 21736v2 Announce Type: replace-cross Abstract: Translation equivariance is a central reason convolutional neural networks have been successful in computer vision.
While recent advancements like the Poincaré ResNet have demonstrated the potential of learning visual representations directly in hyperbolic space, their optimisation remains hampered by the computationally intensive nature of Riemannian gradients and the strict boundaries of the manifold. Furthermore, standard hyperbolic networks treat spatial transformations of the same object as distinct hierarchical concepts, leading to redundant parameter usage and vanishing signals.
arXiv:2606. 17961v1 Announce Type: cross Abstract: Positional encoding is a fundamental component of Transformer architectures, as it injects information about the spatial or sequential arrangement of inputs.