arXiv Machine Learning
Oct 12

Kernelized Parameter Sensitivity Atlases for Amortized Conditional Interpretability

The paper introduces Kernelized Parameter Sensitivity Atlases (KPSA), a method that uses a block decomposition of the empirical neural tangent kernel to estimate input‑specific parameter sensitivities efficiently. KPSA embeds parameter groups in a query space and applies a lightweight kernel to predict sensitivities for unseen inputs, achieving a high fraction of the perturbation susceptibility of full back‑propagation while dramatically reducing computation and memory usage. Experiments across vision, forecasting, and protein‑modeling tasks show that KPSA identifies sparse, depth‑spanning sensitivity circuits and improves throughput and memory efficiency in activation‑attribution problems.

By Davide Wiest, Ahanaf Hasan Ariq, Abdul Monaf Chowdhury
arXiv Computer Vision
Aug 31

Cut-ViT: Task-Specific Model Pruning via Gram Anchoring Subspace Consistency

Cut‑ViT introduces a task‑specific pruning pipeline for visual foundation models that uses gram anchoring matrices and subspace decomposition to align feature representations between native and pruned DINOv3 models. The method incorporates basis‑agnostic and residual constraints to preserve robustness across spatial and channel dimensions, and employs spectral entropy adaptation to tailor the pruning objective to downstream tasks. Experiments demonstrate that Cut‑ViT achieves state‑of‑the‑art performance on six tasks across nine datasets while reducing pruning time to about one minute on a single A100 GPU, using only 20.9% of the time and 45.5% of the GPU memory compared to prior methods.

By Jianjian Yin, Liulei Li, Tao Chen, Yi Chen, Yazhou Yao, Wenguan Wang