arXiv Machine Learning September 22, 2026 By Mohammed Ahnouch, Lotfi Elaachak

Whitening Inverts the Hierarchy: What the Norm of a Whitened Embedding Measures

Read the original on arXiv Machine Learning →

The paper investigates the use of the squared norm of a whitened foundation‑model embedding as a training‑free likelihood surrogate. It shows that the apparent Gaussianity of whitened coordinates stems from the projection central limit theorem, not from a true joint Gaussian distribution, and that the norm is systematically over‑dispersed compared to a Gaussian reference. The authors explain that whitening reverses the encoder’s spectral hierarchy, concentrating norm contributions in near‑degenerate directions dominated by noise, and propose interpreting the squared norm as a Mahalanobis measure of semantic atypicality rather than a log‑likelihood.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 13

Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads

arXiv:2608. 11661v1 Announce Type: cross Abstract: A multiplicative dual-encoder network computes a real-valued output for a pair of inputs as the inner product of their separate encodings.

By Zijian Zhao, Sen Li

llms multimodal

arXiv Machine Learning

Jun 16

InfoNCE Induces Gaussian Distribution

arXiv:2602. 24012v2 Announce Type: replace Abstract: Contrastive learning has become a cornerstone of modern representation learning, allowing training with massive unlabeled data for both task-specific and general (foundation) models.

By Roy Betser, Eyal Gofer, Meir Yossef Levi, Guy Gilboa

safety

Hugging Face Trending Papers

2d ago

Interpretable intrinsic dimension estimation through componentwise calibration of distance and angle

DANCo (Dimensionality from Angle and Norm Concentration) jointly calibrates nearest-neighbor distance and angular statistics and consistently reaches state-of-the-art accuracy on clean intrinsic-dimen...

benchmarks

arXiv Machine Learning

Sep 18

SAGG: Sample-Adaptive Gradient Gating for Robust Multimodal Learning under Heterogeneous Corruption

SAGG: Sample-Adaptive Gradient Gating for Robust Multimodal Learning under Heterogeneous Corruption proposes a new method for handling sample-heterogeneous corruption in multimodal training. The authors prove that batch-level, sample-agnostic linear estimators with a shared modulation parameter inevitably incur bias, and that a sample-level all-or-nothing gating strategy is the only unbiased approach within a natural estimator class. SAGG implements a binary retain-or-discard decision per sample using an online feature-norm quality test and a truncation mechanism for variance control, and demonstrates convergence to clean-loss stationary points while achieving superior performance over ten existing methods on Kinetics-Sounds and UCF-101 under various corruption scenarios.

By Wentao Zhang, Yifan Zhu, Yutong Zhang, Wentao Mo

multimodal safety

arXiv Computation and Language

Sep 1

A Hub of Short Rows Inflates Intrinsic Dimension Estimation of Token Embeddings

arXiv:2608.29702v1 Announce Type: new Abstract: A token-embedding table holds a hub of short rows near its origin, and we show that this cluster biases what nearest-neighbor intrinsic-dimension (ID)...

By Alexandre Quemy

llms rag

arXiv AI

Jun 16

ALCL: An Adaptive Log-Correntropy Loss for Robust Learning under Non-Gaussian Noise

arXiv:2606. 16050v1 Announce Type: cross Abstract: Robust deep learning under heavy-tailed and impulsive noise remains challenging because conventional losses such as mean squared error (MSE) exhibit unbounded sensitivity to outliers.

By Mainak Kundu, Ria Kanjilal, Ismail Uysal