Self-supervision is a powerful technique for learning visual representations from unlabeled data. Existing techniques primarily adopt a two-stage approach for self-supervised learning (SSL): a pretraining stage on unlabeled data followed by a finetuning stage on labeled data.
arXiv:2603.20111v2 Announce Type: replace Abstract: The Joint-Embedding Predictive Architecture (JEPA) is often seen as a non-generative alternative to likelihood-based self-supervised learning, emph...
The paper introduces ReMILP, a reformulation‑contrastive learning framework that uses self‑supervision from equivalent formulations of mixed‑integer linear programs (MILPs). By distinguishing re‑descriptions and substitutions, the method trains a graph neural network and a hypernetwork to predict how variable embeddings transform under changes of variables, achieving invariance and equivariance without solver‑derived labels. The learned representations prove useful for tasks such as binary solution, constraint activity, and integrality gap prediction, and serve as a strong initialization for fine‑tuning.
arXiv:2608. 04213v1 Announce Type: new Abstract: Existing studies on self-supervised learning for white-box networks typically decouple the derivation of white-box networks via optimization algorithms from self-supervised learning paradigms.
arXiv:2205. 07739v4 Announce Type: replace-cross Abstract: Self-training (ST) is a simple yet effective semi-supervised learning method.
arXiv:2610.00753v1 Announce Type: cross Abstract: End-to-end backpropagation has been the dominant mode of training in deep learning, allowing for the coordination of parameter updates across layers...
arXiv:2605. 23995v2 Announce Type: replace-cross Abstract: Self-supervised learning (SSL) has emerged as a promising paradigm for addressing the annotation bottleneck in medical imaging by learning representations from unlabeled data.
The paper introduces Φ-Omni, a self‑supervised learning framework for computational pathology that disentangles synergistic information across histology, genomics, and clinical reports using Partial Information Decomposition. By employing a Synergistic Information Bottleneck and a ΦID objective, the method suppresses redundant signals while maximizing irreducible cross‑modal synergy, leading to improved few‑shot performance on breast and lung whole‑slide image datasets. The authors demonstrate that Φ-Omni outperforms both supervised and other SSL baselines on eight external tasks.
arXiv:2602. 02381v2 Announce Type: replace Abstract: Joint-embedding self-supervised learning (SSL), the key paradigm for unsupervised representation learning from visual data, learns from invariances between semantically-related data pairs.
arXiv:2607. 02447v1 Announce Type: new Abstract: Recent research has introduced distributed self-supervised learning (D-SSL) approaches to leverage vast amounts of unlabeled decentralized data.
The paper proposes a structured version of the Information Bottleneck (IB) that separates label-relevant structure from within-condition variation using a dual-bottleneck formulation. It introduces a conditional KL term that targets within-condition information, allowing explicit control over nuisance-like variation in learned representations. Experiments demonstrate improved performance in low-data classification and consistent gains on dense prediction tasks.
arXiv:2606. 05109v1 Announce Type: new Abstract: To leverage the full potential of multimodal data, we need representations that go beyond the state-of-the-art alignment and fusion approaches and exploit all cross-modal interactions without sacrificing modality-specific information.