arXiv AI By Mia Lassiter, Brinnae Bent

Defining AI Agents: A Compendium of Criteria, Metrics, and Benchmarks

Read the original on arXiv AI →

The article "Defining AI Agents: A Compendium of Criteria, Metrics, and Benchmarks" surveys the lack of a standard definition for AI agents and organizes this ambiguity into five dimensions: environmental interaction, learning and adaptation, autonomy, goal‑directed behavior, and temporal coherence. It reviews how each dimension has been conceptualized in prior work and compiles the metrics, benchmarks, and evaluation frameworks used to assess them. The authors also introduce the Agent Compendium, a public digital resource that extends these evaluation methods, aiming to provide a common structure for evaluating and comparing agent capabilities across AI systems.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 20

Emergence of Agentic AI: A Review on Evolution, Background, Working Principles, Applications, Adoption Factors, and Future Research Directions

The article reviews the emergence of Agentic AI, covering its evolution, theoretical foundations, working principles, and architectural aspects. It surveys recent scholarly contributions across various domains, highlighting real‑world applications, current research findings, and existing challenges. The review also proposes a framework for stakeholder adoption and outlines future research directions to guide researchers and practitioners.

By AKM Bahalul Haque, Al Amin Islam Ridoy, Mohammad Rayhan, Ivan Porres
arXiv AI
Oct 12

A Baseline Assessment for the Agentic Web Technology Stack

The paper introduces MASTIF, a Multi‑Agent System TestIng Framework, to evaluate how different protocols, frameworks, and libraries combine into an Agentic Web solution. It presents a baseline assessment using 3 large language models, 3 protocols, and 4 agent implementation frameworks across 300 Mind2Web benchmark tasks. The results show that task‑specific technology stacks can improve completion rates, efficiency, and social value stability while reducing verbosity and task time.

By Vagner Figueredo de Santana, Diogo Cortiz
arXiv AI
Sep 28

Agentick: A Unified Benchmark for General Sequential Decision-Making Agents

Agentick is a unified benchmark for sequential decision‑making agents that evaluates RL, LLM, VLM, hybrid, and human agents on 37 procedurally generated tasks across six capability categories, four difficulty levels, and five observation modalities via a single Gymnasium‑compatible interface. It includes a Coding API, oracle reference policies, pre‑built SFT datasets, a composable agent harness, and a live leaderboard. An evaluation of 27 configurations and over 90,000 episodes shows no single approach dominates, with GPT‑5 mini leading overall, PPO excelling in planning and multi‑agent tasks, and the reasoning harness boosting LLM performance by 3‑10×, while ASCII observations outperform natural language.

By Roger Creus Castanyer, Pablo Samuel Castro, Glen Berseth
arXiv Computation and Language
Nov 4

Agent Skill Evaluation and Evolution: Frameworks and Benchmarks

The paper surveys the evolving field of agent skill evaluation and evolution, emphasizing the shift from isolated skill creation to automated, evaluation-driven development. It categorizes skill evolution into four paradigms—execution feedback, trajectory distillation, compression, and reinforcement learning—and analyzes six skill-centric benchmark categories to highlight gaps and trade-offs. The authors outline open research directions for building generalizable, efficient, and verifiably safe skill ecosystems.

By Kexin Ding, Yang Zhou, Can Jin, Feng Tong, Mu Zhou, Dimitris N. Metaxas