← Back to all news
arXiv AI By Zimian Wei, Tianqi Zhang, Hengyue Pan

ACSAN: All-Convolutional Self-Attention Network Integrating Convolutional Operations with Vision Transformer

Read the original on arXiv AI →

The paper introduces ACSAN, an All‑Convolutional Self‑Attention Network that replaces the fully‑connected matrix multiplications of Vision Transformers with convolution operations. Its core module, Dynamic Convolutional Self‑Attention (DCSA), uses multi‑scale dilated convolutions to produce Query, Key, and Value tensors, and combines spatial and channel self‑attention into a single output. Experiments on CIFAR‑10, CIFAR‑100, and ImageNet‑1k demonstrate that ACSAN attains competitive or superior classification accuracy to ViT while using comparable or fewer parameters, and ablation studies confirm the positive contribution of each component.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
5d ago

SAGE: Saliency-Annealed Gated Encoder for Universal Vision Recognition

SAGE (Saliency‑Annealed Gated Encoder) is a Vision Transformer‑based backbone that explicitly leverages saliency, global context, and high‑frequency details through a saliency‑guided architecture, an annealing training paradigm, and a two‑stage loss transition. It achieves state‑of‑the‑art or competitive results on a wide range of dense prediction tasks, including COCO object detection and segmentation, ADE20K segmentation, TinyImageNet classification, and DIOR‑R oriented detection, all using the same backbone with task‑specific decoders. The framework is notable for being the first to deliver multi‑task SOTA performance on Huawei Ascend NPUs.

By Le Yang, Yucheng Huang, Junyu Chen, Qi Dang, Xin Kong, Haifeng Huo, Qiang Zhao
llmscomputer-visionbenchmarks
More like this →
Hugging Face Trending Papers
Aug 11

Grid-Preserving Knowledge Distillation: Transferring Convolutional Inductive Bias to Vision Transformers under Data Scarcity

Vision Transformers underperform convolutional networks when training data is scarce, and distilling convolutional inductive biases from a CNN teacher is an effective remedy that leaves the deployed model unchanged. General-purpose feature distillation, however, transfers little in this setting.

llmsefficiencybenchmarkssafety
More like this →
arXiv Computer Vision
Oct 9

MCL: Meta Convolution Layer

The paper introduces the Meta Convolution Layer (MCL), a dynamic convolution approach that models kernels as input‑conditioned high‑order polynomial functions instead of linear mixtures of basis kernels. MCL generates a single adaptive kernel through a structured polynomial meta‑network, improving training stability and expressivity. Experiments show significant accuracy gains on ImageNet, CIFAR, and fine‑grained classification tasks when MCL is added to ResNet, Wide‑ResNet, Swin, and ViT backbones.

By Naim Reza, Md Al Amin, Ho Yub Jung
llms
More like this →