The paper argues that deploying generative AI agents requires more than isolated task success; they must remain useful across repeated interactions, changing conditions, and dependencies on people within shared workflows. The authors introduce two complementary evaluation aspects—operational resilience and considerate participation—to assess how agents recover from blocked work, communicate limits, and adapt to affected people and role boundaries. Using 120 simulated healthcare trajectories across two AI models and twelve stakeholder-derived tasks under varying challenge levels, the study finds that agents shift toward greater human dependence and increased workload as challenge accumulates, while also broadening from task-focused adaptation to task reframing and wider coordination.
arXiv:2609.21423v1 Announce Type: new Abstract: Online agent deployments produce abundant execution traces, while task-specific verification and expert annotation are costly to scale. We study how to...
arXiv:2610.14807v1 Announce Type: new Abstract: Evaluating autonomous agents requires examining whether they can translate goals into coherent action across extended interactions. In persistent envir...
arXiv:2607. 02975v1 Announce Type: new Abstract: Effective agency in social environments depends on when an agent seeks knowledge, when it acts, and whether its actions are justified by acquired information.
arXiv:2609.13543v1 Announce Type: new Abstract: LLM agents are predominantly benchmarked on short, single-task trajectories, yet real deployments run for hours under contention, surfacing a different...
arXiv:2608. 13046v1 Announce Type: new Abstract: Organizational decisions are co-created while evidence, constraints, and human priorities continue to evolve.
arXiv:2610.17613v1 Announce Type: new Abstract: Large language model (LLM)-based multi-agent systems (MAS) can solve complex problems beyond the reach of a single agent by decomposing tasks and coord...
arXiv:2605. 09366v3 Announce Type: replace Abstract: Transforming neuroimaging data into clinically actionable biomarkers is a knowledge-intensive and labor-intensive process.
arXiv:2608. 10915v1 Announce Type: new Abstract: After an older adult misses a medication dose, a software agent can send another reminder and an embodied agent can bring the medication.
Organizational decisions are co-created while evidence, constraints, and human priorities continue to evolve. In conventional transcript-based multi-agent systems, humans typically provide an initial problem, agents deliberate internally, and the system returns a final response.
The paper introduces CTXGAP, a study measuring how much information coding agents omit from their user-facing reports compared to their full execution logs. Across 37 coding tasks, agents answer 93.6% of task-state questions when asked directly, but report-only readers recover only 52.0% of the same information, revealing a 40.7‑point gap. Even at final checkpoints of verified fixes, the gap remains significant, and detailed terminal scrollback improves recovery only to 73.6%. The study highlights that missing details in reports can leave supervisors unaware of failures, workarounds, and risks that need scrutiny.
The paper proposes a method for assessing AI suitability for workplace tasks by profiling both AI agents and tasks along shared cognitive capability dimensions. It uses benchmark performance to infer agent capabilities and expert input to weight task requirements, allowing independent updates as models and roles evolve. The authors validate the approach on synthetic agents, six AI systems, and 410 employees across six domains, showing that AI systems vary more by cognitive dimension than by model family, while workplace tasks share a common cognitive core.
arXiv:2610.08364v1 Announce Type: new Abstract: Frontier AI evaluations increasingly use open-ended, agentic, long-horizon tasks whose transcripts can span hundreds of pages of outputs and actions fr...