arXiv:2606. 12146v1 Announce Type: cross Abstract: Rotary Position Embedding (RoPE) is widely adopted in Transformer models, yet its extension to high-dimensional domains lacks a unified theoretical formulation.
arXiv:2608.29715v1 Announce Type: cross Abstract: Transformers rely on position embedding mechanisms in long context modeling in most cases. Rotary Position Embedding (RoPE) embeds positional informa...
arXiv:2406. 07049v3 Announce Type: replace-cross Abstract: Understanding spatial relationships across all dimensions is fundamental for intelligent systems.
arXiv:2608.31045v1 Announce Type: new Abstract: Rotational symmetry is one of the most important structural principles in machine learning on 3D data. In applications ranging from physics and materia...
arXiv:2605. 29987v2 Announce Type: replace Abstract: Although multi-scales representation learning enables elastic-dimension embeddings, nested subspaces often suffer from dimensional redundancy and spectral collapse.
Despite the widespread adoption of Vision Transformers (ViTs) and their success across numerous computer vision applications, the fundamental understanding of their dimensional and representational geometry remains relatively underexplored. To address this gap, we introduce Transformer Geometry Observatory (TGO), a systematic framework of experiments and analysis pipelines designed to investigate the representational geometry and dynamics of Vision Transformers.
arXiv:2602. 10099v2 Announce Type: replace Abstract: Leveraging representation encoders for generative modeling offers a path for efficient, high-fidelity synthesis.
arXiv:2606. 03260v1 Announce Type: cross Abstract: Deep learning surrogates for 3D Partial Differential Equations (PDEs) often fail to generalize across geometric transformations because they depend heavily on specific coordinate systems.
arXiv:2606. 31061v1 Announce Type: cross Abstract: Tensor Train (TT) decomposition is a powerful technique for analyzing high-dimensional data.
arXiv:2606. 19249v1 Announce Type: cross Abstract: Despite the widespread adoption of Vision Transformers (ViTs) and their success across numerous computer vision applications, the fundamental understanding of their dimensional and representational geometry remains relatively underexplored.
arXiv:2601. 19179v2 Announce Type: replace Abstract: Autoencoders have long been considered a nonlinear extension of Principal Component Analysis (PCA).
arXiv:2608.29867v1 Announce Type: new Abstract: Autoencoders are widely used for nonlinear dimensionality reduction and manifold learning. While most common implementations rely on both nonlinear enc...
arXiv:2607. 08987v1 Announce Type: new Abstract: Spectral embedding methods are widely used for dimensionality reduction and clustering of high-dimensional datasets with intrinsic low-dimensional structures.