arXiv AI

ZEBRAARENA: A Diagnostic Simulation Environment for Studying Reasoning-Action Coupling in Tool-Augmented LLMs

arXiv Computation and Language
Oct 22

Reasoning-Search-Augmented LLMs: An Operational Taxonomy, Benchmark Analysis and Controlled Evidence Synthesis

The paper reviews and categorizes large language models that combine step‑by‑step reasoning with interactive search, treating retrieval as an action within an iterative agentic loop. It analyzes these systems along four architectural axes—retrieval coupling, controller design, knowledge source, and grounding strictness—while also considering the learning signal as a cross‑cutting factor. The authors conduct controlled ablations and benchmark analyses to clarify how design choices affect search, stopping, and grounding behavior, aiming to provide a unified perspective for future research.

By Biswas Poudel, Nilson Chapagain, Amit Kumar, Xianshun Jiang
arXiv AI
Sep 3

UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agents

UniToolCall introduces a unified framework for tool-use in large language model agents, standardizing toolset construction, dataset generation, and evaluation. The framework aggregates over 22,000 tools and creates a hybrid training corpus of more than 390,000 instances by combining ten public datasets with synthetically generated, structurally controlled trajectories. It models diverse interaction patterns—single‑hop vs. multi‑hop, single‑turn vs. multi‑turn, serial vs. parallel execution—and adds an Anchor Linkage mechanism to enforce cross‑turn dependencies, while converting seven public benchmarks into a common Query–Action–Observation–Answer format for fine‑grained evaluation.

By Yijuan Liang, Xinghao Chen, Yifan Ge, Ziyi Wu, Hao Wu, Changyu Zeng, Wei Xing, Xiaoyu Shen
arXiv AI
Oct 26

Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use

The paper introduces a model‑adaptive definition of tool necessity that accounts for each LLM’s empirical performance, contrasting it with observed tool‑call behavior across four models on arithmetic and factual QA datasets. It finds substantial mismatches—26.5‑54.0% for arithmetic and 30.8‑41.8% for factual QA—between when tools are needed and when they are actually used. By probing hidden states, the authors show that the main source of mismatch lies in the transition from internal cognition (recognizing tool necessity) to action (making a tool call), revealing a knowing‑doing gap in LLM tool use.

By Yize Cheng, Chenrui Fan, Mahdi JafariRaviz, Keivan Rezaei, Soheil Feizi
arXiv AI
Aug 11

TempoBench: Reasoning Execution Without Causal Attribution Is Just Simulation

arXiv:2510. 27544v3 Announce Type: replace Abstract: Current training paradigms, optimized for long-horizon reasoning trace execution, have made Large Language Models (LLMs) excel at pattern matching and forward simulation of reasoning, but underperform at counterfactual causal understanding and reasoning.

By Nikolaus Holzer, William Fishell, Baishakhi Ray, Mark Santolucito