arXiv AI

AI Agents Push Humans Out of the Loop

AI agents are increasingly autonomous, posing significant risks that current designs hinder effective human oversight. The paper argues that oversight is degraded by both design choices and the cognitive decline of users who rely heavily on automation. It calls for prioritizing human cognitive needs in AI agent development, proposing design affordances and protocols to maintain critical judgment and counter skill atrophy.

arXiv AI
Jul 21

Nonuniformity Principle in Human-AI Coworking

arXiv:2607. 16530v1 Announce Type: new Abstract: As generative AI is increasingly applied to automate multi-step and high-stake workflows, human judgment and involvement remain essential for ensuring the quality of AI-generated outputs.

By An Luo, Jie Ding
arXiv AI
Aug 14

Humans are Missing from AI Coding Agent Research

arXiv:2608. 12355v1 Announce Type: cross Abstract: Recent progress in AI coding agent research has led to rapid improvements in agents' ability to autonomously perform complex software engineering tasks, from editing large codebases to executing long-horizon development workflows.

By Zora Z. Wang, John Yang, Kilian Lieret, Alexa Tartaglini, Valerie Chen, Yuxiang Wei, Zijian Wang, Lingming Zhang, Karthik Narasimhan, Ludwig Schmidt, Graham Neubig, Daniel Fried, Diyi Yang
arXiv AI
Oct 7

A Case Study in Assuring AI-Written Software

The article reports a case study of a production healthcare platform that was built using software‑engineering agents and overseen by an operator lacking formal software training. Over time the system evolved into a meta‑agent workflow where one agent wrote code, other agents supervised and reviewed it, and project rules carried lessons forward. The study found that tests, monitors, and reviewing agents were fallible—some measured proxies instead of outcomes, audits failed silently, checks disappeared from reports, and an automated repair caused operational disruption—highlighting that human control relied on aligning the intended outcome, the evidence used to judge it, the agents’ permissions, and the final human decision around the same objective.

By Lindsey Ferris, Sierra Bonilla