arXiv Machine Learning
Jul 8

Drift Happens: An Empirical Study of Neural Architecture Robustness to Temporal Distribution Shift

arXiv:2607. 05908v1 Announce Type: new Abstract: Real-world data distributions evolve over time, inducing temporal distribution shift that can substantially degrade the reliability of deployed machine learning systems.

By Robin Holzinger (Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, USA), Riccardo Colletti (Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, USA)
arXiv Machine Learning
Oct 9

HAND: A Biologically-Inspired Activation Function that Improves Generalisation and Sample Efficiency in Image Classification

The paper introduces HAND, a biologically-inspired activation function that incorporates homeostasis, accelerating nonlinearity, and divisive normalization to act as an inductive bias in deep neural networks. Experiments on image classification show that using HAND allows a ConvNeXt-tiny model to reach ImageNet1k accuracy in 25 epochs versus 200 epochs for the baseline, and yields larger accuracy gains on long-tailed and reduced-data settings. The authors report that HAND does not degrade generalisation on common corruptions and can improve the model’s ability to reject unknown classes, with benefits observed across multiple CNN architectures and datasets.

By Michael W. Spratling, Heiko H. Sch\"utt
arXiv Computer Vision
Aug 25

GradAttn: Replacing Fixed Residual Connections with Task-Modulated Attention Pathways

GradAttn replaces fixed residual connections in deep ConvNets with attention‑controlled gradient pathways, allowing the network to dynamically weight shallow texture features and deep semantic representations. The method extracts multi‑scale CNN features at different depths and regulates them through self‑attention, leading to improved performance over ResNet‑18 on five of eight evaluated datasets, including a +11.07% accuracy gain on FashionMNIST. Analysis of gradient flow shows that controlled instabilities introduced by attention can coincide with better generalization, while positional encoding proves to be dataset‑dependent.

By Soudeep Ghoshal, Himanshu Buckchash
arXiv AI
Jun 9

A Mechanistic Analysis of Adversarial Fine-tuning of Vision Transformers

arXiv:2606. 07593v1 Announce Type: cross Abstract: The widespread use of image classification models in high-risk, real-world situations necessitates making these models robust to slight disturbances or perturbations, such as blurring or sharpening, in the input images.

By Hannah Gao (Massachusetts Institute of Technology), Isha Agarwal (Massachusetts Institute of Technology), Dylan Hadfield-Menell (Massachusetts Institute of Technology), Rachel Ma (Massachusetts Institute of Technology)