The paper argues that explainable AI (XAI) tools, intended to increase trust in machine learning models, often fail to do so and instead foster overreliance on incorrect predictions. It attributes this failure to the unfaithfulness of XAI methods, which oversimplify model internals and misalign explanations with the actual model behavior. The authors identify five major shortcomings in current faithfulness metrics and propose four research directions to improve their reliability.
By Marco Zullich, Emily Schiller, Ivan Gentile, David Dembinsky, Adriano Lucieri, Kilian G\"oller, Steffen Seitz
arXiv:2310.16295v2 Announce Type: replace-cross
Abstract: Neural network have achieved remarkable successes in many scientific fields. However, the interpretability of the neural network model is sti...
By Zhimin Li, Shusen Liu, Kailkhura Bhavya, Peer-Timo Bremer, Valerio Pascucci
arXiv:2504.00470v3 Announce Type: replace
Abstract: To develop a trustworthy AI system, it is essential to understand its behavior using attribution methods, which aim to identify the input regions t...
By Ruoyu Chen, Siyuan Liang, Jingzhi Li, Shiming Liu, Li Liu, Hua Zhang, Xiaochun Cao
arXiv:2610.17898v1 Announce Type: cross
Abstract: Sparse autoencoders decompose language model representations into interpretable features, but feature interpretation in this literature remains predo...
By Samuel Xie, Noura Al-Moubayed
arXiv:2511.15371v3 Announce Type: replace
Abstract: Assessing the importance of individual features in Machine Learning is critical to understand the model's decision-making process. While numerous m...
By Eddie Conti, \'Alvaro Parafita, Axel Brando
arXiv:2606. 10877v1 Announce Type: new Abstract: Occlusion-based attribution methods provide an intuitive way to estimate feature importance by perturbing input features and measuring the resulting change in model output.
By Thodoris Lymperopoulos, Ioannis Kakogeorgiou, Denia Kanellopoulou