arXiv AI
Jul 7
Auto: The AGI Compiler
arXiv:2607. 04542v1 Announce Type: cross Abstract: Every LLM agent run re-derives its behavior token by token on a frontier model: brilliant, expensive, slow, and unbounded.
When to Compile a Computer-Use Agent? Measuring Payback and Making Compilation Decisions for Token Efficiency
Related stories
arXiv AI
6d ago
TokenCast: Forecasting Token Consumption During LLM Agent Execution
arXiv:2609.35760v2 Announce Type: replace-cross Abstract: When a large language model (LLM) agent executes the same task, token consumption can vary by over an order of magnitude across runs. The age...
arXiv AI
Aug 5
TraceCompiler: Skill-Guided Mining and Compilation of LLM Agent Traces into Mostly Deterministic Workflows
arXiv:2608. 02680v1 Announce Type: cross Abstract: Tool-using language-model agents repeatedly rediscover procedures they have already executed, producing traces that mix reusable structure with retries, exploration, accidental ordering, and repeated lookups.
arXiv AI
5d ago
Evaluating Agents Across Runtime Contracts: When Mismatch Costs Efficiency or Quality
arXiv:2603.01209v3 Announce Type: replace Abstract: In CodeAct, language-model agents write Python that calls tools and use execution feedback to choose actions. Persistent runtimes preserve Python v...
arXiv AI
Jul 15
Do AI Agents Know When a Task Is Simple? Toward Complexity-Aware Reasoning and Execution
arXiv:2607. 13034v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly automate multi-step engineering and informatics workflows, yet they rarely ask how much effort a task actually requires.
arXiv AI
Sep 23
LazyAgent: Demand-Driven Materialization and Physical Optimization of Agentic Programs
arXiv:2609.23058v1 Announce Type: new Abstract: Current agent runtimes that plan before acting generally execute a step once it becomes ready. We present LazyAgent, a unified execution framework for...
arXiv AI
Sep 2
UniACE: A Unified Framework for Evaluating LLM Agentic Capabilities
UniACE is a unified framework that standardizes the evaluation of large language model (LLM) agents by representing each benchmark as an instruction–tool–environment triplet and running models through a shared, task‑agnostic harness in isolated runtimes. It preserves native success criteria, offers an offline mode for dynamic‑resource tasks, and standardizes efficiency metrics, execution records, and failure attribution. Applying UniACE to 7 benchmarks across 24 domains and 15 models revealed significant score shifts, ranking reversals, and sensitivity to evidence representation, highlighting the impact of evaluation configuration on reported agent performance.
arXiv AI
Aug 18
AstronOS: A Unified Execution Model and Runtime for Long-Horizon Agentic Systems
arXiv:2608. 16381v1 Announce Type: new Abstract: Agentic systems often organize execution and state around a single conversation, model invocation, or agent instance, even when real work spans many calls and stages.
arXiv AI
Jun 17
PreAct: Computer-Using Agents that Get Faster on Repeated Tasks
arXiv:2606. 17929v1 Announce Type: new Abstract: Computer-using agents drive real software through the screen -- clicking and typing -- but they solve every task from scratch: asked to repeat a task, an agent re-reads the screen, re-reasons every tap, and pays the full cost again.
arXiv AI
Aug 18
StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling
arXiv:2608. 15089v1 Announce Type: new Abstract: Long-horizon agents can fail even when their underlying models can solve the constituent steps.
arXiv AI
Aug 26
Beyond Executable Models: The Pufibara Agent Harness and the Modelica Agent Workflow Benchmark for Physical System Modeling
The paper introduces Pufibara, an agent harness designed to maintain engineering state and evidence across revisions in Modelica-based physical system modeling. It also presents a 232-task Modelica Agent Workflow Benchmark covering model repair, generation, and tuning, evaluated by an external benchmark-owned evaluator. Experiments show Pufibara outperforms Claude Code in task success and resource efficiency across two LLM backends.
arXiv AI
Sep 18
BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software
BuildBench introduces a realistic benchmark for evaluating large language model agents on the task of compiling open‑source software (OSS). It includes diverse OSS projects that lack clear build instructions, have undocumented dependencies, and may require source patching or script modification. The authors also present OSS‑BUILD‑AGENT, a baseline LLM‑based agent that retrieves build instructions effectively and achieves state‑of‑the‑art performance on the benchmark.