AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,743 stories · RSS feed

arXiv Computation and Language
Sep 22

CCTU: A Benchmark for Tool Use under Complex Constraints

CCTU is a new benchmark designed to evaluate large language models (LLMs) on their ability to use tools under complex constraints. It includes 200 test cases that average seven constraint types and 4,700‑token prompts, covering resource, behavior, toolset, and response dimensions. An executable validation module performs step‑level checks, and nine state‑of‑the‑art LLMs were tested, revealing that none exceed a 20% task completion rate when strict constraints are enforced, with frequent violations and limited self‑refinement.

By Junjie Ye, Guoqiang Zhang, Wenjie Fu, Zelin Li, Tao Gui, Qi Zhang, Xuanjing Huang
arXiv Machine Learning
Sep 22

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

The paper introduces Regularized Recursive Self-Improvement of Agent Harnesses (RRSI), a method that applies regularization principles to the iterative editing of an LLM agent’s harness—prompts, control flow, tooling, memory, and context management. RRSI limits the number of edits per candidate, encourages novel trajectories, and uses a critic and pruner to filter out benchmark‑specific or ineffective changes, thereby favoring reusable agent mechanisms. Experiments on eight benchmarks show RRSI improves performance by up to 14.1 points on the training split and 4.7 points on out‑of‑distribution tests, while reducing policy token usage by 30% compared to unregularized evolution.

By Peng Xia, Rujun Han, Zifeng Wang, Yanfei Chen, Yufan Zhang, Yoonho Lee, Chengsong Huang, Han Yu, Zhongying CuiZhu, Yifei Ming, Huaxiu Yao, Burak Gokturk, Tomas Pfister, Chen-Yu Lee
arXiv Machine Learning
Sep 22

A Channel-Boosted Multi-Agent System with Iterative Consultation for Document Sensitivity Classification

The paper introduces a Channel‑Boosted Multi‑Agent System (CB‑MAS), instantiated as IC‑MAS, to classify document sensitivity without the input‑length truncation problem of standard transformers. IC‑MAS uses a Channel Critic Agent to adaptively weight two first‑window encoders and Consultation Agents to exchange belief states, achieving 90.72% accuracy and 91.23% F1‑score while reducing computation by ~54% compared to a fixed‑round baseline. The authors provide explainability via LIME/SHAP, multi‑agent evaluation, and a transparent discussion of limitations.

By Aleesha Zainab, Asifullah Khan, Muhammad Ahmed Khalid, Faheem Ullah Khan