arXiv Machine Learning

In Defense of Cosine Similarity: Normalization Eliminates the Gauge Freedom

arXiv:2602. 19393v2 Announce Type: replace Abstract: Steck, Ekanadham, and Kallus [arXiv:2403.

arXiv Machine Learning

Jul 24

Semantics at an Angle: When Cosine Similarity Works Until It Doesn't

arXiv:2504. 16318v3 Announce Type: replace Abstract: Cosine similarity is a standard comparison rule for learned representations in information retrieval, natural language processing, computer vision, and multimodal learning.

By Kisung You

rag computer-vision nlp multimodal safety

Hugging Face Trending Papers

Jun 28

Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings

The standard way to compare two text embeddings is cosine similarity. Scattered studies report that a different metric does better, but never pin down the geometric condition that decides when, or why.

llms rag fine-tuning

arXiv Machine Learning

Jun 16

Size Doesn't Matter: Cosine-Scored Sparse Autoencoders

arXiv:2606. 15054v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) detect features via inner product, so a feature's activation scales with both its directional alignment and the input's norm.

By Silen Naihin, Lev Stambler

safety

arXiv AI

Jun 3

Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models

arXiv:2606. 02765v1 Announce Type: cross Abstract: Model dimension ($d_{model}$) is a fundamental hyperparameter in transformer language models, yet its role in setting the geometric limits of feature representation remains under-explored.

By Alexander Guha

llms rag diffusion

Hugging Face Trending Papers

Sep 23

Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces

Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direction is equally meaningful, but there is no reason...

llms computer-vision fine-tuning efficiency

arXiv AI

Aug 18

Shape Operator PCA: Curvature-Aware Projections for Geometric Machine Learning

arXiv:2608. 15313v1 Announce Type: cross Abstract: In this paper, we propose SHOPCA (Shape Operator-based Principal Component Analysis), a novel method for unsupervised metric learning and dimensionality reduction that incorporates differential geometric information into the covariance structure of classical PCA.

By Alexandre L. M. Levada

rag benchmarks

arXiv AI

Jun 30

High-Dimensional Concentration and Retrieval Instability in Embedding Spaces: Implications for Retrieval-Augmented Generation

arXiv:2606. 28330v1 Announce Type: cross Abstract: Embedding-based retrieval systems rely on the assumption that geometric proximity in highdimensional representation spaces reflects semantic relevance.

By Ernesto Lopez Fune (DE)

rag safety

arXiv Machine Learning

Jun 5

Efficient Mean Curvature Computation on High-Dimensional Data Manifolds

arXiv:2606. 06329v1 Announce Type: new Abstract: Estimating local mean curvature at each point of a high-dimensional dataset is a key ingredient of geometry-aware machine learning algorithms, such as the Mean Curvature Boundary Points (MCBP) method.

By Alexandre L. M. Levada

arXiv Machine Learning

Sep 24

Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces

arXiv:2609.27988v1 Announce Type: cross Abstract: Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direc...

By Andrew Bond, Ege Erdem \"Ozl\"u, Tuna \c{C}imen, Ilkin Umut Melanlioglu, Tolga Birdal, Erkut Erdem, Aykut Erdem

llms computer-vision fine-tuning efficiency

arXiv Machine Learning

Jun 19

Comparing Linear Probes with Mahalanobis Cosine Similarity

arXiv:2606. 19603v1 Announce Type: new Abstract: Linear probes are widely used in interpretability research and often compared by cosine similarity.

By Zhuofan Josh Ying, Peter Hase, Nikolaus Kriegeskorte

safety

arXiv AI

Jun 3

$\mathbb{R}^{2k}$ is Theoretically Large Enough for Embedding-based Top-$k$ Retrieval

arXiv:2601. 20844v3 Announce Type: replace-cross Abstract: This paper studies the Minimal Embeddable Dimension (MED): the least dimension in which there exists a configuration of $m$ object vectors so that every subset of size at most $k$ is exactly retrieved by score comparison.

By Zihao Wang, Hang Yin, Lihui Liu, Hanghang Tong, Yangqiu Song, Ginny Wong, Simon See

llms rag

arXiv Machine Learning

Sep 3

Exact Limits of Random Projections for Preserving Geometry: Distance Recovery, Nearest-Neighbor Rankings, and Covariance Shape in Gaussian Models

arXiv:2609. 02155v1 Announce Type: new Abstract: The Johnson-Lindenstrauss (JL) lemma guarantees that a random projection of $n$ points to $m=O(\varepsilon^{-2}\log n)$ dimensions preserves pairwise squared distances within relative error $\varepsilon$ with high probability, and this dimension order is asymptotically optimal.

By Piyush Sao

Related stories