arXiv:2607. 02447v1 Announce Type: new Abstract: Recent research has introduced distributed self-supervised learning (D-SSL) approaches to leverage vast amounts of unlabeled decentralized data.
Self-supervision is a powerful technique for learning visual representations from unlabeled data. Existing techniques primarily adopt a two-stage approach for self-supervised learning (SSL): a pretraining stage on unlabeled data followed by a finetuning stage on labeled data.
The paper investigates whether pretrained image models can generalize to unseen datasets by clustering their embeddings. Using encoders trained only on ImageNet‑1k, both supervised and self‑supervised, the authors evaluate clustering performance on out‑of‑domain images. They find that supervised encoders perform better within the training domain, while self‑supervised encoders excel far outside it, and that fine‑tuning self‑supervised models reverses this trend. Additionally, the study shows that the silhouette score in UMAP‑reduced space correlates strongly with clustering accuracy, offering a proxy metric when labels are unavailable.
arXiv:2505. 09854v3 Announce Type: replace Abstract: As end-user device capability increases and demand for intelligent services at the Internet's edge rises, distributed learning has emerged as a key enabling technology for the intelligent edge.
arXiv:2606. 04399v1 Announce Type: new Abstract: In the paradigm of decentralized learning, a group of agents collaborate to train a global model using distributed datasets without a central server.
One-shot federated learning (OSFL) addresses the communication overhead of federated learning by limiting training to a single round, but doing so without sacrificing model quality is non-trivial, particularly when client data distributions diverge. Recent work has addressed this challenge by aggregating client knowledge on the server through the construction of transferable synthetic datasets or distillates.
arXiv:2607. 07565v1 Announce Type: cross Abstract: One-shot federated learning (OSFL) addresses the communication overhead of federated learning by limiting training to a single round, but doing so without sacrificing model quality is non-trivial, particularly when client data distributions diverge.
The paper explores contrastive self‑supervised learning (SSL) for retinal fundus image classification, comparing SimSiam and SimCLR under limited data and computational resources. It investigates how retinal‑specific augmentation strategies and training parameters affect representation quality, evaluated through linear probing and fine‑tuning on multi‑disease classification and diabetic retinopathy grading tasks. Results indicate that tailored augmentations enable lightweight SSL models to learn transferable representations, reducing reliance on large annotated datasets while achieving competitive performance.
arXiv:2602. 02381v2 Announce Type: replace Abstract: Joint-embedding self-supervised learning (SSL), the key paradigm for unsupervised representation learning from visual data, learns from invariances between semantically-related data pairs.
AdaDim introduces a training strategy for self‑supervised learning that adaptively balances dimensionality increase and mutual information reduction. By gradually regularizing the projection head while encouraging feature decorrelation and sample uniformity, AdaDim achieves up to 3% performance gains over standard SSL baselines without relying on costly techniques such as queues or predictor networks. The method demonstrates that optimal SSL models do not simply maximize dimensionality or minimize mutual information, but find a trade‑off between the two.
arXiv:2505. 19699v2 Announce Type: replace-cross Abstract: Federated Learning (FL) is a decentralized machine learning paradigm that enables clients to collaboratively train models while preserving data privacy.
The paper explores contrastive self‑supervised learning (SSL) for retinal fundus image classification, comparing SimSiam and SimCLR under limited data and computational resources. It investigates how retinal‑specific augmentation strategies and training parameters affect representation quality, evaluated through linear probing and fine‑tuning on multi‑disease classification and diabetic retinopathy grading tasks. The results demonstrate that tailored augmentations enable lightweight SSL models to learn transferable representations, reducing reliance on large annotated datasets while achieving competitive performance.
arXiv:2608. 02250v1 Announce Type: new Abstract: Federated learning (FL) is a popular distributed learning framework where multiple clients perform local training and a server aggregates the locally updated models.