arXiv Computer Vision
Sep 4

Using Deep Learning Models Pretrained by Self-Supervised Learning for Protein Localization

The study evaluates self‑supervised learning (SSL) models pretrained on ImageNet‑1k and the Human Protein Atlas (HPA) Field‑of‑View (FOV) for protein localization in microscopy images. DINO‑based Vision Transformer backbones pretrained on either dataset transfer well to the OpenCell dataset, achieving strong performance even without fine‑tuning and improving further when fine‑tuned (0.704 ± 0.027 macro F1 on 17 classes). At the single‑cell level, the HPA‑pretrained model outperforms others in k‑nearest‑neighbor classification across all neighborhood sizes (macro F1 ≥ 0.515).

By Ben Isselmann, Dilara G\"oksu, Heinz Neumann, Andreas Weinmann
arXiv Computer Vision
2d ago

Promise and Limits of Cross-Architecture Knowledge Distillation for Lightweight Medical Vision Transformers (LiteMed-ViT)

arXiv:2610.33520v1 Announce Type: new Abstract: Running deep models for medical image analysis at the point of care is challenging with standard architectures: convolutional neural networks (CNNs) ar...

By Shuaibu Shehu (SR University), Elijah M. Flomo (SR University), Mohammad Safiya (GITAM University), Anurodh Kumar (SR University), Subodh Kumar (Dr. KNS Memorial Institute of Medical Sciences)
arXiv Machine Learning
Oct 9

HAND: A Biologically-Inspired Activation Function that Improves Generalisation and Sample Efficiency in Image Classification

The paper introduces HAND, a biologically-inspired activation function that incorporates homeostasis, accelerating nonlinearity, and divisive normalization to act as an inductive bias in deep neural networks. Experiments on image classification show that using HAND allows a ConvNeXt-tiny model to reach ImageNet1k accuracy in 25 epochs versus 200 epochs for the baseline, and yields larger accuracy gains on long-tailed and reduced-data settings. The authors report that HAND does not degrade generalisation on common corruptions and can improve the model’s ability to reject unknown classes, with benefits observed across multiple CNN architectures and datasets.

By Michael W. Spratling, Heiko H. Sch\"utt
Hugging Face Trending Papers
Aug 11

Grid-Preserving Knowledge Distillation: Transferring Convolutional Inductive Bias to Vision Transformers under Data Scarcity

Vision Transformers underperform convolutional networks when training data is scarce, and distilling convolutional inductive biases from a CNN teacher is an effective remedy that leaves the deployed model unchanged. General-purpose feature distillation, however, transfers little in this setting.