Chronicle introduces a method called cut‑point replay to make regression testing of large language model (LLM) agents reproducible. It records an agent run at non‑deterministic boundaries as immutable envelopes and replays selected boundaries while executing the rest live, turning recorded failures into continuous‑integration tests. Benchmarks show minimal overhead, bit‑stable full replay, and effective detection of faulty code changes in a mutation study.
arXiv:2608. 08239v1 Announce Type: new Abstract: LLM routers promise efficiency by matching each request to the cheapest adequate model, and are increasingly applied per step inside multi-step agents.
By Ashritha Gonuguntla
arXiv:2609. 04875v1 Announce Type: cross Abstract: Long-running LLM agents are stateful: beyond the transcript they accrete compressed summaries, plaintext memory, pending tool plans, and, under every serving API, a KV cache.
By Chao Yao, Yangbo Wei, Zhen Huang, Junhong Qian, Chenle Chen, Shaoqiang Lu, Chen Wu, Lei He
arXiv:2610.16333v1 Announce Type: new
Abstract: As AI agents increasingly act through external tools, tool results are no longer simple outputs but execution artifacts that must remain correctly asso...
By Yutong Zhi, Jie Zhang, Xiuqi Lu, Jiayuan Xu, Jing Shao
arXiv:2607. 02255v1 Announce Type: new Abstract: Memory for a long-horizon LLM agent is a contract about what each future decision is allowed to see.
By Xiangchen Cheng, Yunwei Jiang, Jianwen Sun, Zizhen Li, Chuanhao Li, Xiangcheng Cao, Yihao Liu, Fanrui Zhang, Li Jin, Kaipeng Zhang
arXiv:2608. 05784v1 Announce Type: new Abstract: Computer-use agents pay full frontier inference to re-derive routines their user has already performed, because an agent's memory today records what the user said, not what the user did.
By Nossa Iyamu
TestJack is a scalable framework that audits large language model (LLM) coding agents by generating new tests for each trial, focusing on prompt requirements that may be violated. It keeps only the tests passed by the ground‑truth patch and re‑examines failures, providing replayable evidence for each confirmed error. In experiments across six model backends and five benchmarks, TestJack found that 34.4% of trials deemed correct actually violated task requirements, reducing the overall resolution rate from 50.6% to 33.2%.
By Shuangjie Yao, Hao Wang, Koushik Sen, Simin Chen, Baishakhi Ray, Dawn Song
The paper introduces BSC‑R, a deterministic effect‑boundary mechanism that ties a single‑use commit authorization to the specific action and the semantic state that justified it, aiming to close the proposal‑to‑commit gap in tool‑using language‑model agents. Experiments on 2,847 AgentDojo episodes and 10,302 frozen proposals show that BSC‑R preserves the agent’s original behavior while rejecting unauthorized changes, and further tests on a boundary‑drift experiment and the CONTINUITY suite demonstrate high success rates in valid contexts and robust handling of replay and ambiguous cases. However, broader testing reveals that BSC‑R still allows a 25% invalid‑effect commit rate in a larger attack set, indicating that it provides scoped, not universal, safety.
By Wesley Shu
The paper proposes a new reporting standard for attributing agent failures to their history, insisting on intervention‑verified evidence rather than post‑hoc taxonomies. It outlines a method to freeze an executable mid‑trajectory state, alter only the agent’s perceived history, and continue execution to obtain a measured replay certificate. The authors review existing manipulation classes and highlight that while several methods perform interventions, none currently report a quantified replay certificate, and they provide two cautionary examples illustrating the pitfalls of lacking such verification.
By Egor Pakhomov, Erik Nijkamp
The paper presents a small coverage‑closure agent that verifies data integrity for LLM‑generated functional tests in hardware design. It reads coverage from simulation reports (cocotb‑coverage YAML) and uses separate scoreboards to check data, rather than relying on the model’s account. Applied to three different RTL designs—a FIFO, a UART transmitter, and an AHB2APB bridge—the agent increased coverage from 91.7%, 88.9%, and 67.4% to 100% within eight iterations, and also uncovered a clock‑domain‑crossing bug in a manual testbench.
By Venkata Krishna Kumar Vedantam
The paper introduces rebuild‑dossier, an open‑source tool that locks an application’s real interface before code is written and enforces one‑test‑at‑a‑time building through automated checks. In experiments, a compliant agent failed a held‑back test while a rule‑breaking agent passed, showing that a passing test suite can be gamed. The study also demonstrates that the automated check mechanism, rather than interface‑locking alone, is crucial for reliable rebuilds, and that multi‑level verification catches errors that single‑level checks miss.
By Parker Fawcett
arXiv:2607. 28871v1 Announce Type: cross Abstract: When a repair agent runs a test and sees it pass, the result is treated as evidence about the reported defect.
By Xiaonan Xu, Wenjing Wu