arXiv:2606. 26617v1 Announce Type: new Abstract: Scaling laws describe how learning performance varies with model size, data size, and compute.
arXiv:2609.33741v3 Announce Type: replace-cross Abstract: Inverse Optimal Transport (OT) based methods for representation learning learn representations such that the global OT coupling between a pai...
The paper introduces the concept of internal discrepancy, measuring the gap between the mean representation of positive samples and latent cluster centers in contrastive learning. It derives a generalization bound for downstream supervised tasks that depends on this discrepancy, providing a unifying framework for existing assumptions and motivating a relaxed k‑means loss to reduce it. The study also explores how contrastive learning affects downstream behavior, particularly in class‑imbalanced settings, and establishes properties of the resulting supervised losses.
arXiv:2602. 24012v2 Announce Type: replace Abstract: Contrastive learning has become a cornerstone of modern representation learning, allowing training with massive unlabeled data for both task-specific and general (foundation) models.
The paper investigates whether supervised contrastive learning objectives produce a regular simplex geometry in representation space when trained on long‑tailed data. It shows that existing objectives fail to achieve this geometry because they depend on class sample counts. The authors propose a fully balanced contrastive objective that provably forms a regular simplex, leading to a more balanced representation space and improved performance on long‑tailed recognition and transfer learning tasks.
arXiv:2605. 24316v3 Announce Type: replace Abstract: Mini-batching is central to large-scale optimization, yet its role in statistical scaling laws remains limited.
arXiv:2512. 22088v3 Announce Type: replace-cross Abstract: The scaling law, a cornerstone of Large Language Model (LLM) development, predicts improvements in model performance with increasing computational resources.
arXiv:2610. 23337v1 Announce Type: new Abstract: Pretrained models are often reused across downstream tasks, motivating the question of whether this transfer learning procedure can provide a substantial performance improvement, namely, in terms of the scaling law.
arXiv:2605. 07914v2 Announce Type: replace Abstract: Sharpness-aware and gradient-alignment methods have been shown to improve generalization, however each family of methods targets a single geometric property of the loss landscape, while ignoring the other.
arXiv:2402.11215v4 Announce Type: replace Abstract: The choice of batch size in minibatch stochastic gradient optimization is critical for both optimization and generalization performance in large-sc...
arXiv:2609.38784v1 Announce Type: new Abstract: In probabilistic contrastive learning, a shared temperature is commonly interpreted as a shared similarity scale, but this interpretation does not hold...
arXiv:2608.09523v2 Announce Type: replace Abstract: Stochastic gradient descent (SGD) and its variants train deep neural networks (DNNs) effectively in practice, yet classical optimization theory doe...
arXiv:2606. 18650v1 Announce Type: new Abstract: As Large Language Model (LLM) datasets scale to trillions of tokens, data selection has emerged as a critical frontier to filter out uninformative noise and construct adaptive learning trajectories.