arXiv:2606. 30573v1 Announce Type: new Abstract: We introduce SWE-Interact, a new testbed for evaluating coding agents on multi-turn, interactive, user-driven software engineering tasks.
The paper introduces Tool Primitives, a design that replaces rigid API schemas with natural language interfaces for tool calling, enabling seamless inter-tool communication. It builds ToolFace, a repository of over 25,000 functions that LLMs can dynamically retrieve, and HEART, a harness engineering framework that orchestrates tool use with planning, routing, and verification. Experiments show HEART outperforms fine‑tuned models and leading commercial LLMs while cutting API costs by up to 85%.
arXiv:2608. 10039v1 Announce Type: new Abstract: Agentic workflows have become an important abstraction for building reliable LLM-based automation systems by organizing large language models (LLMs), tools, and control logic into explicit execution structures.
arXiv:2608. 10366v1 Announce Type: new Abstract: Real-world data science involves long-horizon workflows that span data wrangling, exploration, modeling, visualization, and validation, and require coordinated use of tools such as notebooks, IDEs, terminals, browsers, and databases within real operating environments.
arXiv:2607. 16617v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to automate data-processing workflows, yet coding agents typically produce scripts that are not automatically materialized as persistent, editable platform artifacts.
Terminal-Universe is a framework that transforms large collections of terminal‑based agent trajectories into reusable, executable environments. By replaying recorded file operations and completing missing files, it reconstructs the original workspace and task, then synthesizes new tasks and multi‑round interactions. The resulting 37.3k task‑sufficient environments enable significant performance gains when fine‑tuning language models on terminal‑based benchmarks.
arXiv:2608.24650v1 Announce Type: cross Abstract: System-level simulation is an essential tool for exploring the rapidly expanding design space of LLM serving systems, where real deployments remain c...
The paper introduces Pufibara, an agent harness designed to maintain engineering state and evidence across revisions in Modelica-based physical system modeling. It also presents a 232-task Modelica Agent Workflow Benchmark covering model repair, generation, and tuning, evaluated by an external benchmark-owned evaluator. Experiments show Pufibara outperforms Claude Code in task success and resource efficiency across two LLM backends.
Terminal-Universe is a framework that converts large collections of terminal-based agent trajectories into reusable, executable environments. By replaying recorded file operations and filling missing files with a completion agent, it reconstructs the original workspace and generates new tasks, scaling them in breadth (cross-workspace queries) and depth (multi-round interactions). The resulting 37.3k task-sufficient environments enable significant performance gains when fine‑tuning language models on terminal‑centric benchmarks.
arXiv:2606. 00288v1 Announce Type: new Abstract: Large language models are undergoing a transition from model technology to system technology.
The paper introduces KNOWS, a benchmark for evaluating web agents that act as assistants by retrieving, synthesizing, and presenting information across complex, multi-step browser tasks. It outlines a task design rubric, evaluation protocol combining deterministic checks with LLM judgments, and reports that current agents achieve only modest success, with the best performing agent succeeding on less than 3% of tasks. The study highlights significant gaps in agents’ tool use, visual understanding, and long‑horizon reasoning.
arXiv:2606. 29823v1 Announce Type: cross Abstract: The database community has repeatedly advanced the state of the art by recognizing that new workloads demand new system architectures.