arXiv AI September 11, 2026 By Yanfei Hu Fleischhauer, Alona Zharova, Nadja Klein, Stefan Feuerriegel

XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?

XAI-Arena proposes using large language models (LLMs) as judges to evaluate the quality of explainable AI (XAI) explanations, aiming for reproducibility, scalability, and multidimensional assessment. The framework assesses dimensions such as simplicity, clarity, task adequacy, trust calibration, actionability, transparency, faithfulness, and overall interpretability across different datasets, models, and stakeholder personas. Human validation shows a strong positive correlation between LLM-generated and human ratings (Spearman's rho = .693, p < .001), supporting the viability of LLM-based evaluations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 17

Towards a Unified Multidimensional Explainability Metric: Evaluating Trustworthiness in AI Models

arXiv:2607. 14315v1 Announce Type: cross Abstract: In this paper, we present a comprehensive framework for assessing the explainability of various XAI methods, such as LIME and SHAP, across multiple datasets and machine learning models, with the ultimate goal of creating a unified multidimensional explainability score.

By Georgios Makridis, Georgios Fatouros, Athanasios Kiourtis, Dimitrios Kotios, Vasileios Koukos, Dimosthenis Kyriazis, Jonh Soldatos

rag benchmarks

arXiv Computation and Language

Sep 7

From Plausible to Actionable: A Position on LLM Self-Explanations

The paper discusses how Large Language Models can produce natural language self‑explanations that appear plausible but may not accurately reflect the model’s reasoning. It critiques current evaluation methods for such explanations and offers practical guidelines to assess their plausibility and faithfulness. Additionally, it argues that evaluation should also consider the actionability of these explanations, showing how they can aid decision‑making for various stakeholders.

By Elize Herrewijnen, Benedetta Muscato, Gizem Gezici, Fosca Giannotti

llms

arXiv AI

Sep 10

Explaining AI Agents Through Execution Traces

arXiv:2609.06063v1 Announce Type: new Abstract: AI Agents are increasingly deployed in real-world settings, where they interact with external tools and make sequential decisions with limited human ov...

By Vittoria Vineis, Fabiano Veglianti, Lorenzo Antonelli, Claudia Di Carlo, Matteo Silvestri, Gabriele Tolomei

llms agents benchmarks

arXiv AI

Aug 3

A Human-Centered Validation of the Explainability-Performance Coefficient

arXiv:2607. 29614v1 Announce Type: cross Abstract: The rapid adoption of deep learning models in high-risk domains has intensified the need for trustworthy Explainable Artificial Intelligence (XAI).

By Christian Oliva, Luis F. Lago-Fern\'andez

efficiency

arXiv AI

Jul 13

All Explanations are Wrong, But Many Are Useful: Exploring the Rashomon Explanation Set with Large Language Models

arXiv:2607. 09502v1 Announce Type: cross Abstract: Explaining machine-learning models is increasingly important for decision-making and consumer trust, yet it is widely believed to come at a cost: existing Explainable AI (XAI) methods suffer from a persistent accuracy-explainability trade-off.

By Pan Li

llms agents benchmarks

Hugging Face Trending Papers

Jun 9

Generative Explainability for Next-Generation Networks: LLM-Augmented XAI with Mutual Feature Interactions

As artificial intelligence and machine learning (AI/ML) models become integral to network operations, their lack of transparency poses a significant barrier to operator trust. Existing explainable artificial intelligence (XAI) techniques often fail to bridge this gap for non-specialists, producing technical outputs that are difficult to translate into actionable insights.

llms benchmarks

One email a morning, machine-written

Related stories

Towards a Unified Multidimensional Explainability Metric: Evaluating Trustworthiness in AI Models

From Plausible to Actionable: A Position on LLM Self-Explanations

Explaining AI Agents Through Execution Traces

A Human-Centered Validation of the Explainability-Performance Coefficient

All Explanations are Wrong, But Many Are Useful: Exploring the Rashomon Explanation Set with Large Language Models

Generative Explainability for Next-Generation Networks: LLM-Augmented XAI with Mutual Feature Interactions