arXiv Computation and Language

Do Chess Explanations Reflect Model Decisions? Behavioral and Token-Level Tests of LLM Reasoning Faithfulness

arXiv AI
Aug 6

Hallucinations on the Board: Tool-Augmented Evaluation of LLM Chess Commentary

arXiv:2608. 04240v1 Announce Type: cross Abstract: Superhuman game engines in domains like chess have made expert-level evaluations easily accessible, yet they communicate what is true without the natural-language explanations that make such expertise educationally useful to experts and non-experts alike.

By S. Ashwin Hebbar, Peiyao Sheng, Sewoong Oh, Pramod Viswanath
arXiv Computation and Language
Sep 7

From Plausible to Actionable: A Position on LLM Self-Explanations

The paper discusses how Large Language Models can produce natural language self‑explanations that appear plausible but may not accurately reflect the model’s reasoning. It critiques current evaluation methods for such explanations and offers practical guidelines to assess their plausibility and faithfulness. Additionally, it argues that evaluation should also consider the actionability of these explanations, showing how they can aid decision‑making for various stakeholders.

By Elize Herrewijnen, Benedetta Muscato, Gizem Gezici, Fosca Giannotti
arXiv AI
Jun 2

From Features to Actions: Explainability in Traditional and Agentic AI Systems

arXiv:2602. 06841v4 Announce Type: replace Abstract: Over the last decade, Explainable AI has primarily focused on interpreting individual model predictions, producing post-hoc explanations that relate inputs to outputs under a fixed decision structure.

By Sindhuja Chaduvula, Jessee Ho, Kina Kim, Aravind Narayanan, Ahmed Y. Radwan, Mahshid Alinoori, Muskan Garg, Dhanesh Ramachandram, Shaina Raza