← Back to all news
arXiv AI By Joonhyeok Jeong, Jy-yong Sohn, Kibok Lee

Rethinking Representation Geometry in Supervised Contrastive Learning under Long-Tailed Data

Read the original on arXiv AI →

The paper investigates whether supervised contrastive learning objectives produce a regular simplex geometry in representation space when trained on long‑tailed data. It shows that existing objectives fail to achieve this geometry because they depend on class sample counts. The authors propose a fully balanced contrastive objective that provably forms a regular simplex, leading to a more balanced representation space and improved performance on long‑tailed recognition and transfer learning tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 26

Internal Discrepancy for Contrastive Learning: Generalization Bounds and Downstream Behaviors

The paper introduces the concept of internal discrepancy, measuring the gap between the mean representation of positive samples and latent cluster centers in contrastive learning. It derives a generalization bound for downstream supervised tasks that depends on this discrepancy, providing a unifying framework for existing assumptions and motivating a relaxed k‑means loss to reduce it. The study also explores how contrastive learning affects downstream behavior, particularly in class‑imbalanced settings, and establishes properties of the resulting supervised losses.

By Jialiang Wang, Qing Ma, Xiong Zhou, Gangfeng Hu, Haoliang Li, Xianming Liu
More like this →
arXiv Machine Learning
Sep 11

Divergence-Based Similarity Function for Multi-View Contrastive Learning

The paper introduces a divergence-based similarity function (DSF) for multi-view contrastive learning, representing each set of augmented views as a distribution and measuring similarity via distribution divergence. DSF captures joint structure across all views, outperforming prior pairwise methods on tasks such as kNN classification, linear evaluation, transfer learning, and distribution shift. It also offers greater efficiency and eliminates the need for a temperature hyperparameter, unlike cosine similarity.

By Jaehyoung Jeon, Cheolsu Lim, Myungjoo Kang
fine-tuning
More like this →