arXiv AI
Jul 7
Builder, Defender, Breaker: The Case Against Removing the Human from the AI-Driven Security Lifecycle
arXiv:2607. 03215v1 Announce Type: cross Abstract: Artificial intelligence has spread across the whole of the security lifecycle.
Builder, Defender, Breaker: Measurable Independence and Bounded Autonomy When Generative Models Build, Defend and Test Software
The article discusses how generative models increasingly act as builders, defenders, and breakers of software, challenging the assumption that full autonomy is the ultimate goal. It introduces a framework that defines measurable independence between lifecycle roles based on shared generative substrates, and proposes five autonomy levels, three human roles, and five decision criteria to guide oversight. The authors argue that human authority should focus on specification, accountability, and emergency intervention, and they outline testable hypotheses and protocols to evaluate independence and oversight effectiveness.
Related stories
arXiv Machine Learning
Oct 8
Restricting the Model, Missing the System: Measurement and Accountability in Offensive AI Governance
arXiv:2605.09504v3 Announce Type: replace-cross Abstract: We show that the instruments used to measure AI offensive capability fail in two ways: (i) they overstate harm, and (ii) they credit the mode...
arXiv AI
Oct 31
Agentic process evidence
The article discusses how AI agents are transforming software development by autonomously creating pull requests, selecting dependencies, refactoring code, interpreting test failures, and managing infrastructure across the SDLC. It highlights the lack of accountability for these transient processes, noting that traditional tools like git logs and governance frameworks assume human decision‑making, which no longer applies. To address this, the authors propose Agentic Process Evidence (APE), a signed, tamper‑evident, machine‑readable attestation that links AI agent sessions to their outcomes, enabling policy‑as‑code controls and human oversight only when necessary.
arXiv AI
Sep 23
From Capability to Assurance in Autonomous Penetration-Testing Harnesses: A Framework and Reference Implementation
arXiv:2609.22664v1 Announce Type: cross Abstract: Research on large language model agents for penetration testing is evaluated almost entirely by capability: whether the agent captures a flag or repr...
arXiv AI
Aug 28
Agent Mesh: Reliability Primitives for Non-Idempotent Agent Delegation - Identity Adequacy and Evidence Adequacy
The paper reports a failure study of a production agentic software‑delivery platform, analyzing 147 incidents across 81 runs. It shows that the standard reliability primitives—retry, timeout, and error‑rate circuit breaking—fail in practice, leading to costly loops, false trips, and blocked work. The authors identify two cross‑cutting causes—identity adequacy and evidence adequacy—and propose seven new reliability primitives that enforce reliability at the delegation level.
arXiv AI
Aug 28
Five Primitives for Governing Autonomous AI Agents at Runtime
The paper proposes five runtime primitives—discovery, identity, governance, attestation, and supply chain—to manage autonomous AI agents in enterprise settings. It argues that traditional control models fail because agents are transient, model-driven, and self‑discoverable, making runtime governance essential. The authors detail an implementation that mediates agent actions against policy, authorizes them via a per‑tenant vocabulary, and records them in a verifiable ledger, noting the associated operational costs and partial deployment status.
Hugging Face Trending Papers
Oct 29
Agentic process evidence
The article discusses how AI agents are transforming software development by autonomously handling tasks across the SDLC, yet their actions are fleeting and lack traceability. It highlights the emerging security risks—intent mismatch, injection, and supply‑chain vulnerabilities—and the absence of standards for agentic provenance and governance. To address this, the authors propose Agentic Process Evidence (APE), a signed, tamper‑evident, machine‑readable attestation that links AI agent sessions to outcomes, enabling policy‑as‑code control and human oversight only when necessary.
arXiv AI
Sep 11
Introducing Consort: A Spec-First Agent Framework for Enforced, Test-Driven Development on Live Database Branches
Consort is a spec‑first, test‑driven agent framework that enforces engineering discipline through immutable controls, a deterministic orchestrator, and human‑approved gates. It operates on live database branches, guiding role agents through a spec‑first design lane and a test‑driven build lane. The framework claims that enforcing tests and gates in code keeps agent‑written code honest and verifiable, while specialized roles make it maintainable.
arXiv AI
Oct 27
Responsibility Boundaries for Agentic Systems: A Comparative Evaluation of Probabilistic Agentic Programming
The paper introduces Probabilistic Agentic Programming (PAP), a framework that explicitly defines responsibility boundaries for language‑model agents by linking mandate, authority, effects, and evidence. It compares PAP to various programming paradigms and agent systems across seven dimensions, and presents a mediation kernel that enforces checks before tool execution, operation‑bound approval, and resource limits. An exploratory study with seven business cases and multiple models demonstrates that PAP’s native Touba implementation achieves the highest or tied observation counts, supporting its effectiveness in managing agentic behavior.
arXiv AI
Jun 15
Regulating the Machine Contributor: Governance and Policy Alignment in Open Source
arXiv:2606. 14594v1 Announce Type: cross Abstract: AI-assisted software development has moved from line-level autocomplete to agents that can plan changes, edit files, and submit pull requests with limited human supervision.
arXiv Machine Learning
Jun 25
The Unfireable Safety Kernel: Execution-Time AI Alignment for AI Agents and Other Escapable AI Systems
arXiv:2606. 26057v1 Announce Type: cross Abstract: AI agents are granted access to tools, APIs, and other infrastructure, making them active principals in those systems.
arXiv AI
Sep 25
The Gold in Bias: Maturing the AI Design Process through Verification
The paper proposes rethinking bias in AI as a diagnostic tool rather than merely a flaw to be minimized. It introduces a multidimensional framework that examines bias across origin, lifecycle emergence, technical causes, and validation methods, covering 30 bias types, 16 verification methods, and 20 countermeasures for both traditional and generative AI. The authors present a hierarchical evidence framework distinguishing internal and external validity, and advocate for Ethics by Design principles to embed bias verification throughout the AI development lifecycle.