ViTAMINS is a method that incorporates synthetic hard negatives into unsupervised vision transformer pretraining to enhance representation quality. The approach is evaluated on ImageNet and a range of downstream tasks—including transfer learning, image retrieval, copy detection, and image/video segmentation—showing significant performance gains. The synthetic negatives also lead to emergent properties, such as representations that encode explicit semantic information and act as strong classifiers, improving over baselines by up to 11.3%.
arXiv:2608. 14922v1 Announce Type: cross Abstract: Mechanistic interpretability has recently expanded to Vision Transformers (ViTs), with Sparse Autoencoders (SAEs) increasingly used as post-hoc tools to decompose internal representations into sparse and more interpretable features.
MLLMCLIP introduces a heterogeneous distillation framework that transfers multimodal knowledge from a generative Multimodal Large Language Model (MLLM) teacher directly into a discriminative CLIP student, eliminating the need for synthetic hard negatives. The method uses an attention-based per-layer token selection and a CKA-based distillation loss to bridge architectural differences between the two models. As a result, MLLMCLIP achieves state‑of‑the‑art compositional accuracy and improves zero‑shot classification and image‑text retrieval performance.
arXiv:2609.06967v1 Announce Type: cross Abstract: Ensuring effective transfer learning for vision-language models without compromising their generalization performance is crucial. However, many exist...
Pretrained vision-language models such as CLIP excel at zero-shot recognition but often fail at compositionality, particularly attribute-object and relational structures. Recent studies mitigate this...
The paper investigates the effect of using semantic positive pairs—different instances of the same class—in self‑supervised visual representation learning. By creating matched ImageNet‑1K subsets of augmented pairs and manually curated semantic pairs, the authors compare contrastive and non‑contrastive SSL methods under identical training conditions. Across transfer learning and object detection tasks, semantic‑pair pretraining consistently outperforms augmented‑pair pretraining, with contrastive methods like SimCLR showing the largest gains, indicating that semantic pairs foster additional invariances beyond standard augmentations.
arXiv:2610.09700v1 Announce Type: new Abstract: Synthetic hard negatives generated in the representation space have proven effective for unimodal self-supervised learning, but transferring this idea...
CoViT introduces a self‑supervised framework that enhances Vision Transformers with instance‑aware representations by leveraging geometry‑guided contrastive learning. It refines attention maps to generate instance masks and constructs triplets that mine the hardest intra‑ and inter‑instance examples, driving a contrastive loss that reduces intra‑instance variance while increasing inter‑instance margins. The method yields consistent AP gains of over 2 points on instance‑level tasks without requiring extra decoders or labels.
arXiv:2603.02767v4 Announce Type: replace-cross Abstract: Image--text contrastive pretraining has become a dominant paradigm for visual representation learning, yet existing methods often yield repre...
The paper introduces ACSAN, an All‑Convolutional Self‑Attention Network that replaces the fully‑connected matrix multiplications of Vision Transformers with convolution operations. Its core module, Dynamic Convolutional Self‑Attention (DCSA), uses multi‑scale dilated convolutions to produce Query, Key, and Value tensors, and combines spatial and channel self‑attention into a single output. Experiments on CIFAR‑10, CIFAR‑100, and ImageNet‑1k demonstrate that ACSAN attains competitive or superior classification accuracy to ViT while using comparable or fewer parameters, and ablation studies confirm the positive contribution of each component.
arXiv:2609.40347v1 Announce Type: new Abstract: We introduce VideoMSN, a Masked Siamese Network framework for efficient self-supervised spatio-temporal representation learning in videos. Instead of r...
The paper explores contrastive self‑supervised learning (SSL) for retinal fundus image classification, comparing SimSiam and SimCLR under limited data and computational resources. It investigates how retinal‑specific augmentation strategies and training parameters affect representation quality, evaluated through linear probing and fine‑tuning on multi‑disease classification and diabetic retinopathy grading tasks. The results demonstrate that tailored augmentations enable lightweight SSL models to learn transferable representations, reducing reliance on large annotated datasets while achieving competitive performance.
Learning-State-Aware Dynamic Generative Data Augmentation on Small-Scale Datasets proposes LSADA, a method that constructs a learning state for each sample based on its loss and loss‑decrease rate to determine a sample‑specific augmentation strength. LSADA also introduces a decoupled data augmentation and diffusion fusion strategy that applies strength‑controlled transformations to class‑relevant regions while generating diverse class‑irrelevant regions, progressively fusing them to enhance image diversity while preserving class semantics. Experiments on nine public datasets demonstrate that LSADA outperforms the current state‑of‑the‑art dynamic GDA method by an average of 4.5% on six natural image datasets and 2.5% on three medical image datasets.