arXiv Machine Learning By Tom\'as Pereira, Jo\~ao Vitorino, Eva Maia, Isabel Pra\c{c}a

Evaluating Local Explainability Metrics for Machine Learning Models on Tabular Data

Read the original on arXiv Machine Learning →

arXiv:2605. 27618v2 Announce Type: replace Abstract: Despite the wide use of explainability techniques to attempt to understand the behavior of Artificial Intelligence (AI), the generated explanations may not always be reliable.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Jul 17

Towards a Unified Multidimensional Explainability Metric: Evaluating Trustworthiness in AI Models

arXiv:2607. 14315v1 Announce Type: cross Abstract: In this paper, we present a comprehensive framework for assessing the explainability of various XAI methods, such as LIME and SHAP, across multiple datasets and machine learning models, with the ultimate goal of creating a unified multidimensional explainability score.

By Georgios Makridis, Georgios Fatouros, Athanasios Kiourtis, Dimitrios Kotios, Vasileios Koukos, Dimosthenis Kyriazis, Jonh Soldatos
arXiv AI
6d ago

A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methods

The paper introduces a synthetic ground‑truth framework for evaluating explainable AI (XAI) methods, addressing the lack of reliable evaluation procedures. By using controlled interventions to create datasets where the importance of input components is known, the framework generates ground‑truth explanations that align with the model’s actual decision process. The authors apply this approach to binary images, tabular data, and time series, and find that nine popular XAI methods exhibit significant limitations, underscoring the need for intervention‑based benchmarks.

By Miquel Mir\'o-Nicolau, Francesco Spinnato, Riccardo Guidotti
arXiv Machine Learning
Sep 18

Evaluating Explanation Methods by the Predictors They Induce

The paper proposes a new evaluation test for explanation methods: if an explanation accurately captures how a model uses its features, one should be able to reconstruct the model’s predictions from it. The authors convert explanations into predictors by summing feature effects and assess how well these predictors reproduce the model on unseen data, without any fitting. They apply this test to partial dependence plots, accumulated local effects, SHAP, and LIME across multiple datasets and model families, showing that the best method depends on feature dependence and that some existing quality metrics can favor flawed explanations.

By Jacob Selb{\ae}k, Hugo L. Hammer