GradAttn replaces fixed residual connections in deep ConvNets with attention‑controlled gradient pathways, allowing the network to dynamically weight shallow texture features and deep semantic representations. The method extracts multi‑scale CNN features at different depths and regulates them through self‑attention, leading to improved performance over ResNet‑18 on five of eight evaluated datasets, including a +11.07% accuracy gain on FashionMNIST. Analysis of gradient flow shows that controlled instabilities introduced by attention can coincide with better generalization, while positional encoding proves to be dataset‑dependent.
arXiv:2508. 17254v2 Announce Type: replace-cross Abstract: Higher levels of machine intelligence demand alignment with human perception and cognition.
The paper introduces HAND, a biologically-inspired activation function that incorporates homeostasis, accelerating nonlinearity, and divisive normalization to act as an inductive bias in deep neural networks. Experiments on image classification show that using HAND allows a ConvNeXt-tiny model to reach ImageNet1k accuracy in 25 epochs versus 200 epochs for the baseline, and yields larger accuracy gains on long-tailed and reduced-data settings. The authors report that HAND does not degrade generalisation on common corruptions and can improve the model’s ability to reject unknown classes, with benefits observed across multiple CNN architectures and datasets.
Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine-learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models?
The paper revisits feature inversion via inverse networks, proposing a modular variant that is computationally efficient and can produce more semantically coherent image reconstructions. It applies this method to large‑scale transformer‑based vision models—Detection Transformer, Vision Transformer, Swin Transformer, and Data‑Efficient Image Transformer—examining reconstructions across network depth. The analysis highlights gradual representational changes in Detection and Vision Transformers, systematic differences in preserving contextual shape and fine‑grained details, and varying robustness to color perturbations, while also demonstrating the framework’s broader applicability to other transformer architectures.
The paper introduces the Meta Convolution Layer (MCL), a dynamic convolution approach that models kernels as input‑conditioned high‑order polynomial functions instead of linear mixtures of basis kernels. MCL generates a single adaptive kernel through a structured polynomial meta‑network, improving training stability and expressivity. Experiments show significant accuracy gains on ImageNet, CIFAR, and fine‑grained classification tasks when MCL is added to ResNet, Wide‑ResNet, Swin, and ViT backbones.
The paper introduces ACSAN, an All‑Convolutional Self‑Attention Network that replaces the fully‑connected matrix multiplications of Vision Transformers with convolution operations. Its core module, Dynamic Convolutional Self‑Attention (DCSA), uses multi‑scale dilated convolutions to produce Query, Key, and Value tensors, and combines spatial and channel self‑attention into a single output. Experiments on CIFAR‑10, CIFAR‑100, and ImageNet‑1k demonstrate that ACSAN attains competitive or superior classification accuracy to ViT while using comparable or fewer parameters, and ablation studies confirm the positive contribution of each component.
arXiv:2610.34678v1 Announce Type: cross Abstract: Humans have been shown to use a "straightened" encoding to represent the natural visual world as it evolves in time (H\'enaff et al. 2019). In the co...
arXiv:2602.09009v2 Announce Type: replace-cross Abstract: Scaling network depth has been a central driver behind the success of modern foundation models, yet recent investigations suggest that deep l...
arXiv:2603.02843v2 Announce Type: replace Abstract: Generalisation across image scales remains a fundamental challenge for deep networks, which often fail to handle images at scales not seen during t...
FlowGenesis is a framework that lets neural networks self‑organize their architecture during training by adding or removing neurons based on a local statistic called Representation Flux. Neurons with high Flux are split to preserve function, while low‑Flux, low‑activity neurons are pruned. Experiments on image and text classification show that networks starting from different sizes converge to similar neuron counts and performance, and the method also improves continual learning by reducing parameters for new tasks without sacrificing accuracy.
arXiv:2511. 12723v2 Announce Type: replace Abstract: Deep neural networks typically rely on the representation produced by their final hidden layer to make predictions, implicitly assuming that this single vector fully captures the semantics encoded across all preceding transformations.
arXiv:2610.33416v1 Announce Type: new Abstract: Accurate assessment of Alzheimer's disease stages from MRI is important for understanding disease progression and supporting the development of compute...