arXiv AI
Aug 12
VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?
arXiv:2608. 10875v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly deployed as personal assistants.
VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?
Large language model (LLM) agents are increasingly deployed as personal assistants. Existing evaluations, however, mostly use short, self-contained requests in static environments.
Related stories
arXiv AI
Jun 29
LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks
arXiv:2604. 13072v2 Announce Type: replace-cross Abstract: OpenClaw-style personal assistants extend LLM agents from isolated tool use to open-ended, stateful, and personalized software environments.
arXiv AI
Jul 24
DynamicMCPBench: A Trace-Grounded, Effect-Scored Benchmark for LLM Agents over Live MCP Servers
arXiv:2607. 20531v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly deployed over Model Context Protocol (MCP) servers, yet the benchmarks used to evaluate them score the final answer or a fixed "ground-truth" list of tools, both of which are fragile once the underlying data is live and stateful.
arXiv AI
Aug 6
OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents
arXiv:2608. 05013v1 Announce Type: cross Abstract: LLM agents are increasingly applied to open-ended everyday requests that span work, study, and life.
arXiv AI
Jul 8
Beyond Reactivity: Measuring Proactive Problem Solving in LLM Agents
arXiv:2510. 19771v4 Announce Type: replace Abstract: LLM-based agents are increasingly moving towards proactivity: rather than awaiting instruction, they exercise agency to anticipate user needs and solve them autonomously.
arXiv Machine Learning
Jun 16
MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents
arXiv:2606. 16748v1 Announce Type: new Abstract: Current benchmarks for computer-use agents evaluate models in impersonal environments.
arXiv AI
Jul 7
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
arXiv:2607. 05174v1 Announce Type: new Abstract: Language agents, i.
arXiv AI
Jul 21
ClawBench: Can AI Agents Complete Everyday Online Tasks?
arXiv:2604. 08523v2 Announce Type: replace-cross Abstract: AI agents may be able to assist with emails and documents, but can they reliably complete everyday online workflows on real websites?
arXiv AI
Jun 2
COMAP: Co-Evolving World Models and Agent Policies for LLM Agents
arXiv:2606. 02372v1 Announce Type: new Abstract: Equipping language agents with world models enables them to anticipate environment dynamics and evaluate candidate actions before execution.
arXiv AI
Jun 3
The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios
arXiv:2601. 08173v2 Announce Type: replace Abstract: The rapid evolution of Multi-modal Large Language Models (MLLMs) has advanced workflow automation; however, existing research mainly targets performance upper bounds in static environments, overlooking robustness for stochastic real-world deployment.
arXiv AI
Jun 6
SentinelBench: A Benchmark for Long-Running Monitoring Agents
arXiv:2606. 05342v1 Announce Type: new Abstract: AI agents are increasingly asked to carry out work that spans minutes, hours, or longer.
arXiv AI
Jun 30
CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agents
arXiv:2511. 02734v3 Announce Type: replace Abstract: Current evaluations of Large Language Model (LLM) agents primarily emphasize task completion, often overlooking resource efficiency and adaptability.