The Flow has not summarised this story yet — read it at arXiv Computer Vision.
arXiv:2603. 18846v3 Announce Type: replace-cross Abstract: Foundation models are used to extract transferable representations from large amounts of unlabeled data, typically via self-supervised learning (SSL).
The study evaluates EfficientNet variants B0–B7 for classifying retinal diseases from spectral-domain optical coherence tomography (SD‑OCT) images at a fixed 224×224 resolution. Eight ImageNet‑pretrained models were tested on the Kermany dataset, with training involving a frozen‑backbone head followed by partial fine‑tuning. Results show non‑monotonic accuracy, with B1 achieving the highest overall accuracy (96.95%) and best Drusen F1, while B7 offered a secondary accuracy gain at the cost of larger model size and CPU latency.
CAMEL (Cross-Architecture Masked Embedding Learning) is a new distillation framework that transfers knowledge from large foundation models to lightweight student models using an asymmetric masking strategy during embedding learning. The method is designed for retinal image analysis, where existing foundation models like RETFound excel but are computationally heavy. Experiments on four datasets demonstrate that CAMEL maintains accuracy while dramatically reducing computational complexity, making it suitable for resource‑limited clinical settings.
arXiv:2606. 15129v1 Announce Type: cross Abstract: Color fundus photography (CFP) is the mainstay for large-scale retinal screening, yet its diagnostic capacity is constrained by the lack of depth-resolved structural information.
The paper revisits the use of contrastive loss for post‑training CLIP models, showing that the reported catastrophic forgetting is mainly due to an overly large temperature parameter in the InfoNCE objective. By setting a smaller temperature, contrastive post‑training can actually improve CLIP. The authors introduce ComCLIP, a lightweight single‑epoch recipe that freezes the text encoder, trains the vision encoder with a properly tempered contrastive loss, an MSE anchoring loss, and a relational distillation loss, achieving comparable zero‑shot classification to CLIP‑Refine while significantly enhancing visual feature transferability and maintaining downstream compatibility with LLaVA.
The paper presents a method for cross‑architecture knowledge distillation from a fine‑tuned DINOv2 Vision Transformer teacher to a lightweight bidirectional Visual State Space Model (LVSSM) student for tea leaf disease classification. By addressing training‑stability issues with a progressive convolutional stem and gated selective‑scan block, the 4.45 M‑parameter student achieves a mean test accuracy of 95.41%—a 3.09‑point improvement over the teacher’s 92.32%—while using only one‑fifth of the teacher’s parameters. Ablation studies show that simple logit‑level distillation outperforms intermediate feature alignment, and the gains are specific to students that start below the teacher’s performance.