arXiv Machine Learning July 3, 2026 By Mohammad Tinati, Stephen Tu

On the Asymptotics of Self-Supervised Pre-training: Two-Stage M-Estimation and Representation Symmetry

Read the original on arXiv Machine Learning →

arXiv:2603. 27631v2 Announce Type: replace Abstract: Self-supervised pre-training, where large corpora of unlabeled data are used to learn representations for downstream fine-tuning, has become a cornerstone of modern machine learning.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

fine-tuning

arXiv Machine Learning

Jul 3

Geometry as a Missing Axis of Representation Quality: The Variational Geometric Information Bottleneck under Data Scarcity

arXiv:2511. 02496v2 Announce Type: replace Abstract: We study latent geometry as an explicit component of representation quality in data-scarce learning.

By Ronald Katende

benchmarks

arXiv Machine Learning

Jul 21

Representation-Aware Distributionally Robust Optimization: A Knowledge Transfer Framework

arXiv:2509. 09371v2 Announce Type: replace-cross Abstract: Distributionally robust optimization (DRO) protects statistical learning against distributional shifts by optimizing the worst-case performance over a set of perturbed distributions.

By Zitao Wang, Nian Si, Molei Liu

fine-tuning

arXiv Machine Learning

Jun 2

Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression

arXiv:2606. 00512v1 Announce Type: new Abstract: In many modern machine learning pipelines, abundant pretrained representations serve as noisy proxy covariates, while task-specific labels remain scarce.

By Kwangho Kim, Jisu Kim

Hugging Face Trending Papers

Jun 1

Provable Data Scaling Law for Meta Learning via Complexity Minimization

Pre-training has become a fundamental paradigm in modern machine learning, with one of its key empirical benefits being reduced downstream sample complexity as the scale of pre-training data increases. However, existing theoretical frameworks for pre-training do not fully explain this phenomenon.

arXiv Machine Learning

Jun 24

Data Augmentation: A Fourier Analysis Perspective

arXiv:2606. 24418v1 Announce Type: new Abstract: Data augmentation is a simple and model-agnostic approach for exploiting known invariances in learning problems.

By Behrooz Tahmasebi, Melanie Weber, Stefanie Jegelka

arXiv Machine Learning

Jul 9

Fast Rates for Semi-Supervised Learning via Data-Augmentation Graph Regularization

arXiv:2607. 07513v1 Announce Type: new Abstract: Self-supervised learning matches supervised accuracy from a fraction of the labels, but the labeled-sample efficiency behind this has lacked a theoretical explanation.

By Adam M. Oberman

safety

Related stories

Geometry as a Missing Axis of Representation Quality: The Variational Geometric Information Bottleneck under Data Scarcity

Representation-Aware Distributionally Robust Optimization: A Knowledge Transfer Framework

Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression

Provable Data Scaling Law for Meta Learning via Complexity Minimization

Data Augmentation: A Fourier Analysis Perspective

Fast Rates for Semi-Supervised Learning via Data-Augmentation Graph Regularization