The paper argues that explainable AI (XAI) tools, intended to increase trust in machine learning models, often fail to do so and instead foster overreliance on incorrect predictions. It attributes this failure to the unfaithfulness of XAI methods, which oversimplify model internals and misalign explanations with the actual model behavior. The authors identify five major shortcomings in current faithfulness metrics and propose four research directions to improve their reliability.
By Marco Zullich, Emily Schiller, Ivan Gentile, David Dembinsky, Adriano Lucieri, Kilian G\"oller, Steffen Seitz
arXiv:2511.15371v3 Announce Type: replace
Abstract: Assessing the importance of individual features in Machine Learning is critical to understand the model's decision-making process. While numerous m...
By Eddie Conti, \'Alvaro Parafita, Axel Brando
arXiv:2606. 06056v1 Announce Type: cross Abstract: Multiple machine learning models can achieve near-equivalent predictive performance on the same task, yet provide divergent feature-based explanations.
By Helge Spieker, J{\o}rn Eirik Betten, Arnaud Gotlieb
arXiv:2608. 02697v1 Announce Type: cross Abstract: Attribution methods (AMs) assign an importance score to each feature and are widely adopted to explain black-box models.
By Eddie Conti, \'Alvaro Parafita, Axel Brando
The paper proposes a new evaluation test for explanation methods: if an explanation accurately captures how a model uses its features, one should be able to reconstruct the model’s predictions from it. The authors convert explanations into predictors by summing feature effects and assess how well these predictors reproduce the model on unseen data, without any fitting. They apply this test to partial dependence plots, accumulated local effects, SHAP, and LIME across multiple datasets and model families, showing that the best method depends on feature dependence and that some existing quality metrics can favor flawed explanations.
By Jacob Selb{\ae}k, Hugo L. Hammer
The paper introduces localized case-based feature importance (CBFI), a framework that decomposes individual predictions in complex tabular models into distinct structural components, separating independent main effects from pure interactions. Unlike additive explanations such as SHAP and LIME, CBFI explicitly captures feature synergies, enabling the recovery of latent interaction structures. Experiments on wine quality classification and medical insurance regression, along with fidelity tests and synthetic benchmarks, show that CBFI accurately disentangles interactions hidden by additive models and aligns with the nonlinear dependencies in gradient boosting and deep neural networks.
By Sejong Oh