The paper examines three open-source agent harnesses—LangChain’s deepagents, Earendil’s pi, and DeepSeek’s dsh—each built from contrasting design philosophies. By analyzing their source code and commit histories, the authors find that the mature harnesses converge on five common architectural elements: a commoditized loop, an append‑only replayable session record, model quirks stored as data, progressive disclosure of context, and explicit extension seams. A fourth harness, used as a held‑out check, also displays all five elements and even reuses another’s implementation, indicating that convergence arises from parallel discovery, diffusion, and literal reuse rather than independent invention. The study notes a missing dimension—external verifiability via a tamper‑evident record—highlighting a future axis for provenance‑sensitive domains.
By Dai Jiahong
The paper introduces VerifierBench, a benchmark comprising 222 tasks from six agent benchmarks and 4,993 hidden test cases designed to evaluate the generation of executable verifiers by large language model agents. It presents Verifier4Verifier (V4V), a six-check audit that screens source verifiers and test suites, and reports that across 19 models, normalized balanced accuracy ranges from 10.6 to 62.1, with generated verifiers more reliably rejecting invalid solutions than accepting valid ones. Case studies highlight that many false rejections stem from verifiers that accept no tested solution, revealing unsupported assumptions about solution formats, tool use, and code changes that exclude valid solutions, and underscore the need to test both acceptance of correct solutions and other permissible solutions.
By Jianing Yin, Ziyi Chen, Hao Wang, Yiyou Sun, Zijian Jin, Dawn Song, Daoan Zhang
arXiv:2605.06445v2 Announce Type: replace-cross
Abstract: Large Language Model (LLM) agents demonstrate strong performance in autonomous code generation under loose specifications. However, productio...
By Francesco Dente, Dario Satriani, Paolo Papotti
arXiv:2607. 14896v1 Announce Type: cross Abstract: Addressing a structural-engineering request requires more than a single answer; it requires a chain of interdependent artifacts: interpreted requirements, a computable model, validation records, solver outputs, code-check records, and a final report.
By Sizhong Qin, Yi Gu, Yao Jiang, Ao Cai, Changjian Zhou, Shaoxuan Shuai, Jiachang Wang, Tianhao Shen, Yueqiang Li, Xinhao Li, Li Zeng, Yueshi Chen, Dachen Gao, Genrong Xu, Wenjie Liao, Xinzheng Lu
The paper proposes adding a verification layer to large language model agents, using a V-model design where each step of the agent’s plan has a dedicated verifier. In a pilot study on the MuSiQue dataset, verified configurations outperformed unverified ones, with deterministic gates correcting most errors at zero cost. The authors plan to extend the architecture to full integration and evaluate it over the next year.
By Ali Afoud, Jie JW Wu
arXiv:2601.19747v2 Announce Type: replace-cross
Abstract: In the rapidly evolving field of Electronic Design Automation (EDA), the deployment of Large Language Models (LLMs) for Register-Transfer Lev...
By Jiale Liu, Taiyu Zhou, Tianqi Jiang