The paper introduces a synthetic ground‑truth framework for evaluating explainable AI (XAI) methods, addressing the lack of reliable evaluation procedures. By using controlled interventions to create datasets where the importance of input components is known, the framework generates ground‑truth explanations that align with the model’s actual decision process. The authors apply this approach to binary images, tabular data, and time series, and find that nine popular XAI methods exhibit significant limitations, underscoring the need for intervention‑based benchmarks.
By Miquel Mir\'o-Nicolau, Francesco Spinnato, Riccardo Guidotti
XAI-Arena proposes using large language models (LLMs) as judges to evaluate the quality of explainable AI (XAI) explanations, aiming for reproducibility, scalability, and multidimensional assessment. The framework assesses dimensions such as simplicity, clarity, task adequacy, trust calibration, actionability, transparency, faithfulness, and overall interpretability across different datasets, models, and stakeholder personas. Human validation shows a strong positive correlation between LLM-generated and human ratings (Spearman's rho = .693, p < .001), supporting the viability of LLM-based evaluations.
By Yanfei Hu Fleischhauer, Alona Zharova, Nadja Klein, Stefan Feuerriegel
arXiv:2606. 21678v2 Announce Type: replace-cross Abstract: Language models can generate plausible rationales for their predictions, but these explanations may not faithfully represent the model's internal reasoning.
By Vatsal Ananthula, Adarsh Kumarappan
arXiv:2610.00601v1 Announce Type: cross
Abstract: Reasoning-enabled VLA policies expose chain-of-thought (CoT) traces that appear to explain and guide their actions, creating a potential interface fo...
By Sathwik Karnik, Joseph JR. Lee, Aryaman Gupta, Somil Bansal
arXiv:2605. 27618v2 Announce Type: replace Abstract: Despite the wide use of explainability techniques to attempt to understand the behavior of Artificial Intelligence (AI), the generated explanations may not always be reliable.
By Tom\'as Pereira, Jo\~ao Vitorino, Eva Maia, Isabel Pra\c{c}a
arXiv:2608. 13760v1 Announce Type: cross Abstract: Which reasoning behaviors are associated with correct answers in reasoning models, and does reasoning-oriented training amplify those behaviors?
By Jean de Dieu Nyandwi, Leena Mathur, Yonatan Bisk, Robert Hawkins, Graham Neubig