The paper investigates whether the tokens highlighted by large language models (LLMs) as key to their predictions are behaviorally consistent when those tokens are manipulated. Using sufficiency, comprehensiveness, and counterfactuality metrics, the authors evaluate closed‑source GPT‑4o‑mini and open‑source models Gemma3, Granite8B, and DeepSeek on IMDB and Steam review datasets. Results show that GPT‑4o‑mini consistently follows expected patterns, Gemma3 and Granite8B maintain sufficiency but falter under stricter interventions, and DeepSeek variants exhibit structural inconsistencies, indicating that explanation reliability varies across model families and sizes.
By Alisson Rosa Pereira, Bruno Iochins Grisci
FOLLIE is a new French natural language inference resource that links sentences to compositional first‑order logic representations, comprising 19,983 inference records and 36,987 sentence‑slot‑formula mappings derived from six heterogeneous NLI sources. The dataset’s silver annotations are generated by a neural‑symbolic pipeline, including 111 LLM‑assisted additions, and the construction integrates morphosyntactic analysis, probabilistic tagging, grammatical proof search, discourse representation, and first‑order semantic construction. An automated inference engine built on Prover9 and Mace4 achieves up to 71.35% accuracy on SICK‑FR, with high agreement rates on committed decisions, illustrating the resource’s utility for studying deductive inference in French.
By Maximos Skandalis
arXiv:2606. 08532v5 Announce Type: replace Abstract: Modern artificial intelligence excels at prediction but cannot explain.
By Lei Lin, Xinlong Pan, Ronghao Wang, Chunbao Zhou, Jue Wang, Yangang Wang, Ivana Rasovska
arXiv:2606. 16603v1 Announce Type: cross Abstract: LLM-based agents have demonstrated strong capabilities in data-intensive analytical tasks, yet their outputs are rarely verifiable: a reliance on linear text trajectories makes their reasoning difficult to audit.
By Jiajie Jin, Zhao Yang, Wenle Liao, Yuyang Hu, Guanting Dong, Xiaoxi Li, Yutao Zhu, Zhicheng Dou
NeSySpec is a neuro‑symbolic audit framework that uses Logic Tensor Networks to evaluate engineering specifications, detect violations, identify altered fields, propose minimal corrections, and link image evidence to parts. It learns tolerances from real records and integrates all audit outputs into a single verification report. The system was validated on 107 aircraft types, achieving superior macro‑F1 scores and higher accuracy in diagnosing constraints and locating altered fields compared to existing methods.
By Pier Mattia Basciano, Gianvito Antenori, Cristiano Andolfi, Giovanni Farina, Salvatore Monteleone
arXiv:2609.39243v1 Announce Type: new
Abstract: Causal interventions such as activation patching and distributed alignment search (DAS) are the main tool for making mechanistic claims about neural ne...
By Beiming Liu, Minjie Chen
arXiv:2602. 02304v3 Announce Type: replace Abstract: Large-scale foundation models exhibit behavioral shifts when subjected to interventions such as scaling, fine-tuning, reinforcement learning with human feedback, or in-context learning.
By Martino Ciaperoni, Marzio Di Vece, Roberto Pellungrini, Luca Pappalardo, Fosca Giannotti, Francesco Giannini
The paper introduces a method for analyzing how interventions in a prompt affect a language model’s reasoning by using behavioral counterfactuals. It trains a module to edit the residual stream so that the next-token distributions for a fixed response match between a source and a counterfactual prompt, outperforming standard activation reconstruction. The resulting attribution maps and visualizations help identify causally important locations for counterfactual recovery and guide hypothesis testing with activation patching, as demonstrated in a case study on Gemma-4-E4B-it and DeepSeek-R1-0528-Qwen3-8B.
By Shawn Im, James Oldfield, Adel Bibi, Sharon Li
arXiv:2607. 26929v1 Announce Type: cross Abstract: The same diagnostic result can support or challenge one causal claim yet fail to address another when the claims concern different populations, outcomes, estimands, pathways, or identifying assumptions.
By Weiyi Kong, Zhuoran Li
arXiv:2608.23370v1 Announce Type: new
Abstract: Large Language Models (LLMs) recognise patterns but do not natively track the path of exclusions that a coherent discourse demands. When an input rests...
By Aldo Gangemi, Emanuele Bottazzi
arXiv:2607. 15495v1 Announce Type: cross Abstract: Out of everything the human brain processes, only a small fraction is consciously accessible, in the sense of being available for verbal report, deliberate control, and flexible reasoning.
By Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul Bogdan, Euan Ong, Rowan Wang, Ben Thompson, David Abrahams, Subhash Kantamneni, Emmanuel Ameisen, Joshua Batson, Jack Lindsey
arXiv:2610.14381v1 Announce Type: new
Abstract: Large language models are increasingly used as scientific assistants, yet we still lack reliable ways to evaluate whether they can investigate a system...
By Yiwei Chen, Amila Weerasinghe, Siddharth Biswal, Utkrisht Rajkumar, Olcay Boz, Taylor Berg-Kirkpatrick, Rose Yu