arXiv AI

CASEL: Consequence-Aware Selective Execution for Reliable Long-Horizon Tool-Using Agents

CASEL (Consequence-Aware Selective Execution Layer) is a runtime allocation policy for tool‑using language agents that chooses among direct execution, verification, sandboxing, clarification, approval, and abstention for each proposed tool call. It combines calibrated failure probability, state‑dependent consequence, reversibility, ambiguity, and evidence freshness to minimize a per‑mode cost. In controlled simulations, CASEL cuts harmful‑side‑effect trajectories from 14.75 % to 7.02 % and reduces mean normalized harm from 0.184 to 0.123 compared to an uncertainty‑ranked gate.

arXiv AI
4d ago

Responsibility Boundaries for Agentic Systems: A Comparative Evaluation of Probabilistic Agentic Programming

The paper introduces Probabilistic Agentic Programming (PAP), a framework that explicitly defines responsibility boundaries for language‑model agents by linking mandate, authority, effects, and evidence. It compares PAP to various programming paradigms and agent systems across seven dimensions, and presents a mediation kernel that enforces checks before tool execution, operation‑bound approval, and resource limits. An exploratory study with seven business cases and multiple models demonstrates that PAP’s native Touba implementation achieves the highest or tied observation counts, supporting its effectiveness in managing agentic behavior.

By Khadim Lo, Soukaina Alaoui Ismaili, Jean-S\'{e}bastien Dessureault
arXiv AI
Oct 5

LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios

LPS-Bench is a benchmark designed to evaluate the safety awareness of computer‑use agents (CUAs) in long‑horizon planning tasks that involve tool workflows. It uses a template‑guided multi‑agent pipeline to generate user instructions, simulated toolkits, and case‑specific safety criteria, followed by human review, allowing scalable expansion without building separate application environments. The benchmark includes 570 cases from 65 scenarios across seven task domains and nine planning‑risk types, and an LLM‑based evaluator assesses tool choices, arguments, and responses throughout execution. Evaluations of 13 LLM agents show persistent safety failures in both benign and adversarial settings, with prompt‑based interventions providing only model‑dependent improvements.

By Tianyu Chen, Chujia Hu, Dongrui Liu, Xia Hu, Wenjie Wang
arXiv AI
Aug 19

Runtime Governance for Agentic AI: Action-Boundary Control with Trusted Provenance and Fail-Closed Execution

The paper introduces Aegis, a runtime governance system for agentic AI that treats model outputs as action proposals and mediates them through a trusted decision layer before tool execution. Aegis evaluates proposals against active policy, resolves provenance server‑side, fails closed under uncertainty, and routes selected cases through a Senate‑style settlement process. In a sandbox evaluation across 6,300 rows, Aegis prevented all governed mock‑tool applications and risky side‑effect completions, preserving provenance and quorum evidence for all settled cases.

By Adam Mazzocchetti
arXiv AI
Oct 2

From Proposal to Verified Effect: Praxa, an Evidence-Bound Harness for Governed AI Agent Execution

Praxa is an evidence‑bound harness for governed AI agent execution that explicitly represents states such as proposal, authority, dispatch, verified external effect, and promotion through deterministic admission, brokered execution, external read‑back, reconciliation, and reviewed promotion. The authors report four evidence lanes: a repository‑local audit passing all unit and Workerd tests; a pilot on 12 curated tasks where both baseline and reliability‑layer arms passed 17 of 36 trials; a coordination‑proxy comparison where both baseline and a source‑authored candidate completed all 180 trials with equal accuracy but the candidate used fewer tokens and steps; and deployed source/configuration evidence showing bounded reflection, recall accounting, memory compilation, and tool‑health paths. None of the evidence demonstrates superiority in security, safety, or user benefit. whyItMatters:"Praxa provides a testable architecture that makes authority‑to‑effect transitions explicit, offering a framework for verifying AI agent behavior, though current evidence does not prove improved security or performance."

By Stefan G. Creadore