arXiv AI
ASIWorld: Benchmarking Autonomous Agents in a Persistent Survival Sandbox
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 21
Harness Self-Evolution for Long-Horizon Terminal Agents
The paper introduces RiseHarness, a framework that iteratively refines agent harnesses while keeping model weights fixed. It uses successful and failed runs under the same configuration to analyze failures, combining agent trajectories with runtime monitor traces to understand interactions between model actions and execution components. The framework then searches other harness frameworks for relevant implementations, adapts them, and validates changes through behavioral checks and repeated task evaluation, ultimately integrating mechanisms from multiple frameworks to improve reliability across tasks.
arXiv AI
Sep 15
Asclepius: An Adaptive Harness for Long-Horizon Clinical Agents
arXiv:2609.13543v1 Announce Type: new Abstract: LLM agents are predominantly benchmarked on short, single-task trajectories, yet real deployments run for hours under contention, surfacing a different...
arXiv AI
Sep 12
Finishing the Task Is Not Enough: Evaluating Agent Resilience and Considerate Participation under Accumulating Challenge
The paper argues that deploying generative AI agents requires more than isolated task success; they must remain useful across repeated interactions, changing conditions, and dependencies on people within shared workflows. The authors introduce two complementary evaluation aspects—operational resilience and considerate participation—to assess how agents recover from blocked work, communicate limits, and adapt to affected people and role boundaries. Using 120 simulated healthcare trajectories across two AI models and twelve stakeholder-derived tasks under varying challenge levels, the study finds that agents shift toward greater human dependence and increased workload as challenge accumulates, while also broadening from task-focused adaptation to task reframing and wider coordination.
arXiv AI
5d ago
Evidence-Guided Recovery for AI Agents with Ambiguous Tool Outcomes and Partial Execution
The paper introduces an evidence‑guided recovery framework for AI agents that encounter ambiguous tool outcomes and partial execution, such as timeouts or failures. It defines a shared admissibility check for key expiry, authorization timing, dependencies, and residual effects, and a terminal certificate to distinguish actual goal satisfaction from publicly supported safe completion. In a finite study of 37 assignments, the approach achieved 28 certified completions, and in a larger comparison of 58 paired units, evidence‑guided advice certified 30 completions versus 9 for evidence prompting, demonstrating a significant improvement in task completion rates.
arXiv AI
Aug 17
AgentRewind: Recoverable Execution for Long-Horizon LLM Agents
arXiv:2608. 14380v1 Announce Type: new Abstract: Many real-world tasks require LLM agents to interact with their environments over long execution horizons.
arXiv AI
Aug 14
Beyond Final Scores: A Systematic Evaluation of Agents for Long-Horizon AI Research and Development
arXiv:2608. 13417v1 Announce Type: new Abstract: Autonomous agents are increasingly capable of improving models, systems, and other technical artifacts through long-horizon experimentation.