arXiv AI

From Actions to Understanding: Conformal Interpretability of Temporal Concepts in LLM Agents

arXiv:2604. 19775v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly deployed as autonomous agents capable of reasoning, planning, and acting within interactive environments.

arXiv AI
Sep 7

DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems

The paper introduces DCFA, a training‑free framework for attributing failures in large language model‑based multi‑agent systems. DCFA uses a global module to build causal‑inspired dependency graphs from system traces, pinpointing the earliest decisive error, and a local module that refines this attribution through counterfactual reasoning. Experiments on the Who&When benchmark across six LLMs demonstrate that DCFA improves step‑level accuracy by up to 8.27% over existing baselines.

By Zehao Wang, Lanjun Wang, Shilong Jin, Junjie Chen, Yanghua Xiao
arXiv AI
Aug 28

From Atomic to Agentic: Towards Interpretable Evaluation of LLMs' Agentic Mathematical Capabilities

The paper introduces a new benchmark that evaluates large language models (LLMs) on their agentic mathematical reasoning rather than just final answers. It aligns problem‑solving behaviors with a taxonomy of reusable mathematical atomic capabilities and includes planning, action, and feedback tasks in both textual and multimodal settings. Experiments show that models with similar end‑to‑end accuracy can have very different agentic profiles, highlighting the importance of process‑level evaluation.

By Jiayi Kuang, Yinghui Li, Yunze Song, Keyu Chen, Zhifeng Shen, Yangning Li, Yidong Wang, Di Yin, Ruizhi Qiao, Xing Sun, Kai Jin, Ying Shen, Liang Lin, Philip S. Yu
arXiv AI
2d ago

ReLiveGym: Evaluating Long-Lived Agents over Weeks of Replayed Reality

ReLiveGym is a diagnostic environment that evaluates long‑lived language‑model agents over weeks of chronologically replayed real‑world streams such as news, market data, and social media. The tasks vary in time sensitivity, reasoning depth, and recurrence, and the study tests eight base language models to see how model choice and harness design—especially action timing—affect performance. Continuous learning from hindsight feedback is also examined to address failure modes in these long‑term tasks.

By Xisen Jin, Jingheng Li, Zhenglun Chen, Junyi Du, Xiang Ren
arXiv AI
Jul 7

Exploring Plan Space through Conversation: An Agentic Framework for LLM-Mediated Explanations in Planning

arXiv:2603. 02070v3 Announce Type: replace Abstract: When automating plan generation for a real-world sequential decision problem, the goal is often not to replace the human planner, but to facilitate an iterative reasoning and elicitation process, where the human's role is to guide the AI planner according to their preferences and expertise.

By Guilhem Fouilh\'e, Rebecca Eifler, Antonin Poch\'e, Sylvie Thi\'ebaux, Nicholas Asher