arXiv:2508. 17254v2 Announce Type: replace-cross Abstract: Higher levels of machine intelligence demand alignment with human perception and cognition.
By Xiao Zhang, Kai-Fu Yang, Xian-Shi Zhang, Hong-Zhi You, Hong-Mei Yan, Yong-Jie Li
The paper introduces HAND, a biologically-inspired activation function that incorporates homeostasis, accelerating nonlinearity, and divisive normalization to act as an inductive bias in deep neural networks. Experiments on image classification show that using HAND allows a ConvNeXt-tiny model to reach ImageNet1k accuracy in 25 epochs versus 200 epochs for the baseline, and yields larger accuracy gains on long-tailed and reduced-data settings. The authors report that HAND does not degrade generalisation on common corruptions and can improve the model’s ability to reject unknown classes, with benefits observed across multiple CNN architectures and datasets.
By Michael W. Spratling, Heiko H. Sch\"utt
Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine-learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models?
The paper revisits feature inversion via inverse networks, proposing a modular variant that is computationally efficient and can produce more semantically coherent image reconstructions. It applies this method to large‑scale transformer‑based vision models—Detection Transformer, Vision Transformer, Swin Transformer, and Data‑Efficient Image Transformer—examining reconstructions across network depth. The analysis highlights gradual representational changes in Detection and Vision Transformers, systematic differences in preserving contextual shape and fine‑grained details, and varying robustness to color perturbations, while also demonstrating the framework’s broader applicability to other transformer architectures.
By Jan Rathjens, Shirin Reyhanian, David Kappel, Laurenz Wiskott
The paper introduces the Meta Convolution Layer (MCL), a dynamic convolution approach that models kernels as input‑conditioned high‑order polynomial functions instead of linear mixtures of basis kernels. MCL generates a single adaptive kernel through a structured polynomial meta‑network, improving training stability and expressivity. Experiments show significant accuracy gains on ImageNet, CIFAR, and fine‑grained classification tasks when MCL is added to ResNet, Wide‑ResNet, Swin, and ViT backbones.
By Naim Reza, Md Al Amin, Ho Yub Jung
The paper introduces ACSAN, an All‑Convolutional Self‑Attention Network that replaces the fully‑connected matrix multiplications of Vision Transformers with convolution operations. Its core module, Dynamic Convolutional Self‑Attention (DCSA), uses multi‑scale dilated convolutions to produce Query, Key, and Value tensors, and combines spatial and channel self‑attention into a single output. Experiments on CIFAR‑10, CIFAR‑100, and ImageNet‑1k demonstrate that ACSAN attains competitive or superior classification accuracy to ViT while using comparable or fewer parameters, and ablation studies confirm the positive contribution of each component.
By Zimian Wei, Tianqi Zhang, Hengyue Pan