arXiv:2608. 12358v1 Announce Type: cross Abstract: Product and engineering teams building role-bearing AI agents face an evaluation gap: an agent can produce accurate, safe, and fluent content while still failing the behavioral requirements of its assigned role.
arXiv:2608. 04148v1 Announce Type: cross Abstract: Agentic AI is increasingly used to coordinate planning, implementation, review, and testing in software development, yet it often offers limited transparency into its decisions and interactions.
The article "Defining AI Agents: A Compendium of Criteria, Metrics, and Benchmarks" surveys the lack of a standard definition for AI agents and organizes this ambiguity into five dimensions: environmental interaction, learning and adaptation, autonomy, goal‑directed behavior, and temporal coherence. It reviews how each dimension has been conceptualized in prior work and compiles the metrics, benchmarks, and evaluation frameworks used to assess them. The authors also introduce the Agent Compendium, a public digital resource that extends these evaluation methods, aiming to provide a common structure for evaluating and comparing agent capabilities across AI systems.
arXiv:2607. 08285v1 Announce Type: new Abstract: Current AI evaluation frameworks focus primarily on technical performance, including accuracy, robustness, reasoning ability, and policy compliance.
arXiv:2607. 21495v1 Announce Type: new Abstract: AI agents are increasingly created inside organizations by non-engineering users through low-code, no-code, and conversational development environments.
arXiv:2607. 14275v1 Announce Type: new Abstract: Context engineering has become central to building reliable AI agents, yet it remains largely unmeasured.
arXiv:2608. 15591v1 Announce Type: new Abstract: Large Language Model (LLM) agents deployed in production environments face a fundamental tension: the agent's behavior is frozen at deployment time, while the business rules and edge cases it must handle continue to evolve.
arXiv:2512. 04123v4 Announce Type: replace-cross Abstract: LLM-based agents already operate in production across many industries, yet we lack an understanding of what technical methods make deployments successful.
arXiv:2607. 10856v1 Announce Type: cross Abstract: The rise of Software Engineering (SE) agents, i.
The paper introduces Agent-Integrated Software (AIS), a pattern that embeds an intelligent agent within an existing application to address persistent coordination challenges. It defines Intent-Level Interaction Abstraction (IIA) as the task semantics for user inspection and control, and presents interaction contracts that constrain the relationship between AIS execution and IIA states. The authors provide a transition-system model, a compact disclosure contract, and conditional propositions to illustrate how local component validity is insufficient and how admission invariants can be separated from planning, concluding with a research agenda for making agent integration a maintainable software engineering discipline.
arXiv:2606. 04321v1 Announce Type: new Abstract: Agentic AI deployments face a recurring design tension: heavy human oversight limits scale, while broad autonomy outruns accountability.
Large language model (LLM) agents are increasingly evaluated on their ability to use tools, plan multi-step tasks, coordinate with other agents, and operate over extended horizons. Reported benchmark gains often obscure recurring failure modes documented across otherwise unrelated evaluation efforts.
The paper titled "Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025" examines how experienced developers employ AI agents in software development. Through field observations and surveys, it finds that developers value agents for productivity but maintain control over design and implementation to ensure quality. They use agents as collaborative tools rather than full delegation, selecting tasks based on suitability and leveraging their expertise to guide agent behavior.