Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation

The paper presents a quantitative study of how well the attention maps of CNNs, Vision Transformers, and Vision Mamba align with human visual perception, using eye‑tracking data as ground truth. A standardized normalization pipeline maps attribution maps and gaze heatmaps to spatial probability distributions, and alignment is measured with Jensen‑Shannon similarity, Chi‑square similarity, and Pearson correlation across fine‑grained classification, medical imaging, and emotion recognition tasks. Results show that CNNs exhibit more global, diffuse saliency that generally matches human perception, ViTs focus on fine‑grained, patch‑level features, and Vision Mamba performs well in medical tasks but diverges in natural images, with architectural and training choices affecting interpretability.