arXiv AI By Junhwa Song, Keumgang Cha, Junghoon Seo

On Pitfalls of $\textit{RemOve-And-Retrain}$: Data Processing Inequality Perspective

Read the original on arXiv AI →

arXiv:2304. 13836v5 Announce Type: replace-cross Abstract: The RemOve-And-Retrain (ROAR) benchmark is widely used to evaluate feature attribution methods, yet its validity remains underexplored from an information-theoretic perspective.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning
Jun 10

XtrAIn: Training-Guided Occlusion for Feature Attribution

arXiv:2606. 10877v1 Announce Type: new Abstract: Occlusion-based attribution methods provide an intuitive way to estimate feature importance by perturbing input features and measuring the resulting change in model output.

By Thodoris Lymperopoulos, Ioannis Kakogeorgiou, Denia Kanellopoulou
arXiv AI
Jul 23

In-Run Data Shapley for Adam Optimizer

arXiv:2602. 00329v4 Announce Type: replace-cross Abstract: Reliable data attribution is essential for mitigating bias and reducing computational waste in modern machine learning, with the Shapley value serving as the theoretical gold standard.

By Meng Ding, Zeqing Zhang, Di Wang, Lijie Hu
arXiv Machine Learning
Sep 21

Data Attribution via Sketched Metadifferentiation

The paper introduces two algorithms, MAGE and SPELL, that enable efficient data attribution in neural networks by estimating a large influence matrix from a limited number of measurements. These methods leverage existing metagradient techniques without additional computational overhead, addressing the challenge of predicting the impact of removing training data in non‑convex models. Experiments show that MAGE and SPELL outperform current baselines across various training scales and measurement budgets.

By Yuxi Chen, Hamza Golubovic, Han Tong, Arian Maleki, Andrew Ilyas
arXiv AI
Sep 16

ResLRP: The Role of Residual Cancellation in Attribution Instability in Vision Transformers

The paper introduces ResLRP, an extension of Layer-wise Relevance Propagation that explicitly handles residual connections in Vision Transformers to prevent attribution explosion. It demonstrates that residual cancellation causes instability in ViT explanations, and that ResLRP improves faithfulness and localization across a wide range of ViT architectures, including Vision Language Models. The method also provides a diagnostic measure for predicting attribution degradation and successfully localizes Sparse Autoencoder features.

By Jim Berend, Reduan Achtibat, Daniel Sch\"affer, Alexander Binder, Wojciech Samek, Sebastian Lapuschkin, Maximilian Dreyer