The paper introduces a new resilience evaluation framework for Embodied Agents Systems (EAS), defining metrics for Rebound, Stability, and Graceful Extensibility. It demonstrates that traditional outcome-centric metrics obscure dynamic behaviors, while the proposed metrics reveal process-level distinctions such as recovery costs and instability across 400 household tasks with 10 EAS. Optimizing with these metrics reduces recovery costs and improves stability and extensibility, highlighting a trade‑off among resilience characteristics that should guide deployment-specific system design.
arXiv:2607. 26121v1 Announce Type: cross Abstract: Embodied intelligence integrates learned perception and decision making with real-time computation, control, and physical interaction.
arXiv:2605. 27898v2 Announce Type: replace Abstract: As LLMs are increasingly deployed as agents, reliable assessment of their agentic capabilities has become essential.
arXiv:2608. 12851v1 Announce Type: new Abstract: Self-improving LLM agents convert successful trajectories into persistent cross-task state.
arXiv:2602. 16666v3 Announce Type: replace Abstract: AI agents are increasingly deployed to execute important tasks.
The paper introduces TraceTrial-Bench, a benchmark that evaluates the reliability of AI agent harnesses across seven components—context, execution, governance, lifecycle, observability, tools, and verification—using 95 expert-level tasks with executable or state-based ground truth. It employs a Trajectory Tracing module to record agent execution, a two-layer evaluation framework to separately assess agent performance and harness reliability, and a Multi-source Evidence Acquisition module to gather evidence for LLM-based evaluation. The benchmark’s analysis of 3,420 runs across four harnesses and nine models shows harness scores between 59.8% and 72.0%, with 31.1% of runs succeeding despite failing at least one harness criterion, highlighting the need for auditable and reproducible harness evaluation.
arXiv:2607. 14275v1 Announce Type: new Abstract: Context engineering has become central to building reliable AI agents, yet it remains largely unmeasured.
arXiv:2607. 18243v1 Announce Type: new Abstract: Agentic AI is crossing trust boundaries faster than current risk models can represent.
arXiv:2609.16305v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly operate over long-horizon interactions involving tool use, persistent state, evolving authorization, and...
arXiv:2607. 03283v1 Announce Type: new Abstract: Embodied intelligence systems require not only end-to-end policy models, but also reusable functional modules that transform multimodal observations, robot states, human demonstrations, and task contexts into structured representations, decisions, trajectories, control references, and system services.
arXiv:2608. 13867v1 Announce Type: cross Abstract: AI coding agents are commonly evaluated as models but deployed as systems.
arXiv:2610.16155v1 Announce Type: new Abstract: Interactive-agent evaluations commonly rely on standardized LLM-based user simulators that exhibit broadly cooperative behavior, potentially overlookin...
The paper argues that deploying generative AI agents requires more than isolated task success; they must remain useful across repeated interactions, changing conditions, and dependencies on people within shared workflows. The authors introduce two complementary evaluation aspects—operational resilience and considerate participation—to assess how agents recover from blocked work, communicate limits, and adapt to affected people and role boundaries. Using 120 simulated healthcare trajectories across two AI models and twelve stakeholder-derived tasks under varying challenge levels, the study finds that agents shift toward greater human dependence and increased workload as challenge accumulates, while also broadening from task-focused adaptation to task reframing and wider coordination.