The paper introduces a work-centric framework for understanding and optimizing work agents, which combine foundation models with runtime harnesses to perform tasks in stateful workspaces. It defines two key dimensions—work persistence and work openness—to characterize working conditions and classifies digital work into five forms based on task scope and organization. The authors identify five core capability families necessary for reliable execution, explore optimization strategies for models and harnesses, and evaluate existing metrics from the perspective of verifiable work outcomes.
As foundation models evolve from information generation to digital work execution, agents are increasingly capable of completing complex tasks across tools, applications, and workflows. However, exist...
The paper introduces WorkForge, a scalable synthesis framework that builds verifiable work‑agent environments from real‑world resources. By extracting factual anchors from expert workflows and associated files, WorkForge generates task instructions, solution plans, and verifiers that remain traceable to observable workspace evidence. The authors constructed 16.7K environments across 40 professional domains, demonstrating significant performance gains for large language models such as Qwen3.5-35B-A3B-Base and Qwen3.5-27B.
The paper proposes a new way to describe benchmarks for AI systems that perform knowledge work, outlining four explicit fields: represented activity, tested setting, required work product, and evaluated result. It builds an inventory of 18 work activities from O*NET to enable activity-level reporting across occupations, and evaluates these activities for semantic coherence, algorithm sensitivity, ontology legibility, and human interpretability. The authors apply their framework to three existing benchmarks—GDPval, OfficeQA Pro, and APEX-SWE—to show how different aspects of work can be captured within the same representation.
arXiv:2608. 05013v1 Announce Type: cross Abstract: LLM agents are increasingly applied to open-ended everyday requests that span work, study, and life.
Harness-Bench is a diagnostic benchmark that evaluates how different harness configurations affect LLM agent performance in realistic workflows. It tests 106 sandboxed offline tasks across multiple model backends, recording artifacts, execution traces, and usage statistics to analyze completion, process quality, efficiency, and failure behavior. The study finds significant variation in outcomes across model-harness pairings, indicating that agent capability should be reported at the configuration level rather than solely by the base model.
The paper introduces KNOWS, a benchmark for evaluating web agents that act as assistants by retrieving, synthesizing, and presenting information across complex, multi-step browser tasks. It outlines a task design rubric, evaluation protocol combining deterministic checks with LLM judgments, and reports that current agents achieve only modest success, with the best performing agent succeeding on less than 3% of tasks. The study highlights significant gaps in agents’ tool use, visual understanding, and long‑horizon reasoning.
arXiv:2610.14966v1 Announce Type: new Abstract: Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evalua...
AgentEvolver is a system that enables agents to develop reusable capabilities while executing tasks, without altering the underlying foundation model. It uses eight entity families to expose operations, methods, agents, control flow, interfaces, and state for revision, coordinated by a shared Runtime and persistent planning. In evaluations on SWE-bench Pro Public, it achieved an 82.08% resolution rate, demonstrating retained capabilities across subsequent website, game, and research tasks, though some objectives remained incomplete.
arXiv:2606. 17819v1 Announce Type: cross Abstract: Agent skills -- structured, reusable knowledge artifacts that augment LLM agent capabilities -- have been rapidly adopted in industry, yet their cross-domain impact and use across commercial and open-source models remain under-studied, and no reusable methodology exists for evaluating an individual skill.
arXiv:2609.23806v1 Announce Type: new Abstract: Many knowledge-work benchmarks are constructed around individual tasks, with the context needed for each task selected together with or after the task...
arXiv:2609.39325v1 Announce Type: new Abstract: The ability of Large Language Model (LLM) agents to complete daily and professional work is receiving increasing attention. Training such agents requir...
CoworkOS is a general‑purpose harness designed for long‑horizon real‑world automation, separating task strategy from execution management. Its three‑layer architecture allows direct execution or delegation to subagents, while a shared application layer lets humans and agents collaboratively edit documents, designs, and 3D scenes. The system incorporates hierarchical workspaces and persistent semantic memory to maintain continuity across agents and sessions, and demonstrates strong performance across 12 benchmarks and six application categories, outperforming leading agents like Codex and DeepSeek Harness.