← Back to all news
arXiv AI By Fouad Bousetouane

EIO-Agents: The Missing Semantic Layer for AI Agent Evaluation

Read the original on arXiv AI →

EIO-Agents introduces an open specification for interoperable AI agent evaluation, consisting of two layers: the Evaluation Intelligence Ontology (EIO) and the Portable Evaluation Record (PER). The EIO layer supplies a semantic framework that defines typed evidence, behavioral predicates, evidence contracts, claims, witness rules, proof status, recurrence, and computable derivations for metrics, findings, controls, and decisions such as PASS, REVIEW, or BLOCK. The PER layer offers a canonical, content‑addressed record of an evaluation that preserves the evidence‑to‑decision chain, enabling re‑derivation, explanation, and verification of the evaluation outcome.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Jun 16

From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents

arXiv:2606. 04990v2 Announce Type: replace-cross Abstract: Large language model (LLM)-based agents are evolving from passive text generators into autonomous systems capable of planning, tool use, retrieval, memory access, environmental interaction, and multi-agent collaboration.

By Yiqi Wang, Jiaqi Zhang, Taotao Cai, Zirui Liu, Qingqiang Sun, Zequn Sun, Zhangkai Wu, Manqing Dong, Mingkai Zhang, Xuefei Yin, Yanming Zhu
llmsagentsbenchmarkssafety
More like this →
arXiv AI
Jun 9

Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting

arXiv:2606. 09809v1 Announce Type: new Abstract: AI evaluation results are produced at scale but reported inconsistently across leaderboards, model cards, benchmark papers, and company blogs.

By Avijit Ghosh, Anka Reuel, Jenny Chim, Wm. Matthew Kennedy, Srishti Yadav, Jennifer Mickel, Yanan Long, Andrew Tran, Anastassia Kornilova, Damian Stachura, Kevin Klyman, Felix Friedrich, Jeba Sania, Max Lamparth, Jan Batzner, Anoop Mishra, Eliya Habba, Yixiong Hao, Nathan Heath, Shalaleh Rismani, Usman Gohar, Andrea Loehr, David Manheim, Ruchira Dhar, Sree Harsha Nelaturu, Aarush Sinha, Leshem Choshen, Drishti Sharma, Ishan Khire, Amit Saha, Subramanyam Sahoo, Michael Hardy, Michael Alexander Riegler, Kabir Manghnani, Michelle Lin, Yanan Jiang, Yilin Huang, Asaf Yehudai, Jessica Ji, Aris Hofmann, Mubashara Akhtar, Nuno Moniz, Yacine Jernite, Stella Biderman, Zeerak Talat, Sanmi Koyejo, Mykel Kochenderfer, Irene Solaiman
benchmarks
More like this →
arXiv AI
Oct 16

AgentEval: Evidence-Bounded Assessment of Stateful Agents

AgentEval is a system that tracks a contract-bound assessment record throughout a stateful agent’s tool execution, capturing state, events, and evidence to determine whether obligations are satisfied, violated, or unresolved. In 32 tool‑world administrations, independent checks confirmed all 30 complete episodes, while controlled interventions on 48 evidence views uncovered 36 binding faults that hash‑only checks missed. A public‑benchmark study on 192 held‑out executions identified ten completed subgoals within broader task failures and highlighted the impact of missing evidence on overall task success.

By Ruiqi Gao (Alibaba Cloud, Alibaba Group)
agentsbenchmarks
More like this →