The paper investigates whether AI agents truly understand computer architecture by comparing their performance when the same 15‑dimensional accelerator design space is presented either as meaningful architectural knobs or as anonymous variables. Using the AutoTuring framework, the authors find that agents with architectural knowledge outperform blind agents on a nine‑kernel FP16 GEMM benchmark, yet a critic loop can largely recover this advantage. The study highlights that architectural knowledge and structured critique act as substitutes rather than complements in improving agent performance.
By Ambika Sharan, Grigory Chirkov, Soheil Abbasloo
The paper introduces $ au^ au$-Bench, a benchmark that turns the construction of AI agents into a measurable task. In this environment a developer agent receives real business records, client requirements, a production API, an existing codebase, and constraints on cost and models, and must deliver a complete customer‑service agent. The benchmark evaluates performance by deploying the agent against simulated users, revealing that current state‑of‑the‑art models achieve only 23.9% success while an expert‑written reference scores 82.2%.
By Quan Shi, Keshav Dhandhania, Karthik Narasimhan, Victor Barres
arXiv:2607. 21268v1 Announce Type: cross Abstract: In many social-science research tasks, such as economics, LLM-based agents must produce outputs for which no cheap, task-complete, machine-readable correctness signal exists.
By Chen Zhu, Xiaolu Wang, Weilong Zhang
arXiv:2607. 13034v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly automate multi-step engineering and informatics workflows, yet they rarely ask how much effort a task actually requires.
By Junjie Yin, Xinyu Feng
The paper introduces the concept of substrate blindness, where AI agents lack execution context in their planning. By providing a 128 MB RAM and 10 s wall‑time contract to large language models, the authors show that agents generate code that uses less memory, runs faster, and incorporates structural changes such as bounded blocking and in‑place buffers. Across three leading models, contract disclosure improved resource usage and correctness, demonstrating that minimal execution contracts can guide agents to produce more efficient programs.
By Manu Agrawal
We study what happens when a single general-purpose large language model acts as the sole researcher on a long-horizon neural architecture design problem. The agent receives a scientific question, an initial hypothesis and motivation, a compute budget, and research affordances (source and experiment management, experiment tracking, literature access, and persistent memory), then autonomously proposes, implements, evaluates, and records experiments over an extended period.
arXiv:2606. 30111v1 Announce Type: cross Abstract: Embodied agents are typically built as hand-designed compositions of perception, memory, planning, and action modules.
By Jian Zhou, Sihao Lin, Jin Li, Shuai Fu, Gengze Zhou, Qi Wu
arXiv:2606. 30111v2 Announce Type: replace-cross Abstract: Embodied agents are typically built as hand-designed compositions of perception, memory, planning, and action modules.
By Jian Zhou, Sihao Lin, Jin Li, Shuai Fu, Gengze Zhou, Qi Wu
arXiv:2607. 02436v1 Announce Type: cross Abstract: Agentic coding assistants are increasingly given extra capabilities, such as browser based testing tools and design oriented system prompts, on the assumption that more capability yields better software.
By Achint Mehta
arXiv:2606. 28279v1 Announce Type: cross Abstract: We present HORIZON, a self-evolving agent framework that treats hardware design as repository-level code evolution.
By Cunxi Yu, Chenhui Deng, Nathaniel Pinckney, Brucek Khailany
Embodied agents are typically built as hand-designed compositions of perception, memory, planning, and action modules. This modularity exposes a large architectural design space, but current systems still rely on researcher intuition to choose where information is stored, how observations are processed, and how model calls are connected.
Eureka is a task‑conditioned Meta‑Agent architecture that transforms long‑horizon scientific tasks into dynamic obligation graphs with explicit acceptance semantics. During execution it constructs Macro‑Agents equipped with specialized state, memory, operators, tools, verifiers, and local topology, using receding‑horizon planning, architecture promotion, and minimal‑sufficient compilation. The system demonstrates strong empirical performance, completing all 170 recursive tasks, generating 3,948 certificates without false acceptances, and achieving significant reductions in input size, recomputation, and consistent serialization across 16,000 concurrent executions.
By Alizer Wong, Heng Cui, Yi Tan, Xiongchao Zhan, Liang Lin, Yuxiang Guo, Zhaorong Dai, Zixin Zeng, Wenyuan Li