arXiv AI By Ambika Sharan, Grigory Chirkov, Soheil Abbasloo

Do AI Agents Understand Computer Architecture?

Read the original on arXiv AI →

The paper investigates whether AI agents truly understand computer architecture by comparing their performance when the same 15‑dimensional accelerator design space is presented either as meaningful architectural knobs or as anonymous variables. Using the AutoTuring framework, the authors find that agents with architectural knowledge outperform blind agents on a nine‑kernel FP16 GEMM benchmark, yet a critic loop can largely recover this advantage. The study highlights that architectural knowledge and structured critique act as substitutes rather than complements in improving agent performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 31

Learning to Formulate: Agentic Hardware-Software Co-Design of AI Infrastructure through Progressive Optimization and Verification

The paper introduces Formulaic, a workflow that separates the tasks of an agentic language-model agent, a mathematical solver, and verification to co-design AI hardware and software. The agent iteratively refines problem formulations from the computation graph down to the chip, using templates that guarantee correctness. Experiments on NVIDIA GB10 and a simulated Ventus GPU show significant speedups for Qwen models and faster verification through proof reuse.

By Yi Ding, Li Shang
arXiv AI
Sep 7

$\tau^\tau$-Bench: An Environment for End-To-End, Realistic Agent Construction

The paper introduces $ au^ au$-Bench, a benchmark that turns the construction of AI agents into a measurable task. In this environment a developer agent receives real business records, client requirements, a production API, an existing codebase, and constraints on cost and models, and must deliver a complete customer‑service agent. The benchmark evaluates performance by deploying the agent against simulated users, revealing that current state‑of‑the‑art models achieve only 23.9% success while an expert‑written reference scores 82.2%.

By Quan Shi, Keshav Dhandhania, Karthik Narasimhan, Victor Barres
arXiv AI
Oct 9

Humanize: Judgement Engineering for Agentic Coding

Humanize is a multi‑agent workflow for agentic coding that uses explicit, mechanically enforced decisions—called judgement engineering—to separate planning, implementation, review, and learning. A human approves a plan contract, a builder agent implements it in rounds, and a reviewer agent from another vendor decides completion, with deterministic hooks and 72 mechanical gates ensuring that a defect survives only if both models miss it. The system has been deployed in over 68 versions in 108 days, earned 1,468 GitHub stars, and powered applications such as a 567‑file gem5 build‑system migration, Kernel Design Agents that excelled in MLSys 2026 contests, and Humanize Olympiad Agents that achieved top scores in multiple international competitions and high rankings on PutnamBench and Lean‑Eval. whyItMatters":"Humanize demonstrates that a structured, multi‑agent approach with explicit mechanical gates can improve reliability and performance in automated code generation, as evidenced by its adoption, star count, and success in competitive benchmarks."

By Sihao Liu, Ligeng Zhu, Zijian Zhang, Dongyun Zou, Zhengyang Zhang, Changye Li, Song Bian, Song Han, Tony Nowatzki
arXiv AI
Sep 7

Substrate-Aware AI Agents: Execution Context as a First-Class Input

The paper introduces the concept of substrate blindness, where AI agents lack execution context in their planning. By providing a 128 MB RAM and 10 s wall‑time contract to large language models, the authors show that agents generate code that uses less memory, runs faster, and incorporates structural changes such as bounded blocking and in‑place buffers. Across three leading models, contract disclosure improved resource usage and correctness, demonstrating that minimal execution contracts can guide agents to produce more efficient programs.

By Manu Agrawal