Hugging Face Trending Papers

Chronicle: Cut-Point Replay for Regression Testing of LLM Agents

Read the original on Hugging Face Trending Papers →

Chronicle introduces a method called cut‑point replay to make regression testing of large language model (LLM) agents reproducible. It records an agent run at non‑deterministic boundaries as immutable envelopes and replays selected boundaries while executing the rest live, turning recorded failures into continuous‑integration tests. Benchmarks show minimal overhead, bit‑stable full replay, and effective detection of faulty code changes in a mutation study.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI
Sep 18

Chronicle: Cut-Point Replay for Regression Testing of LLM Agents

Chronicle introduces a method called cut‑point replay to make regression testing of large language model (LLM) agents reproducible. It records an agent’s run at non‑deterministic boundaries as immutable envelopes and then replays selected boundaries while executing the rest live, enabling continuous‑integration tests that detect faulty code changes. Benchmarks show minimal overhead, perfect bit‑stability, and effective detection of unsafe actions in a mutation study.

By Tisha Chawla, Susheem Koul
arXiv AI
Oct 9

TestJack: Should you trust the results in coding benchmarks? Agentic Coding Benchmarks Auditing via Evaluator Evolution

TestJack is a scalable framework that audits large language model (LLM) coding agents by generating new tests for each trial, focusing on prompt requirements that may be violated. It keeps only the tests passed by the ground‑truth patch and re‑examines failures, providing replayable evidence for each confirmed error. In experiments across six model backends and five benchmarks, TestJack found that 34.4% of trials deemed correct actually violated task requirements, reducing the overall resolution rate from 50.6% to 33.2%.

By Shuangjie Yao, Hao Wang, Koushik Sen, Simin Chen, Baishakhi Ray, Dawn Song
arXiv Computation and Language
Nov 2

Agentic World Modeling for Agent Test-time Scaling without Environment Replica

Agentic World Modeling (AWM) redefines next-observation prediction as a multi-turn evidence-acquisition task, allowing agents to query a real environment safely and construct observations without mutating shared state. Integrated with trajectory-level test-time scaling, AWM enables parallel exploration over a single shared environment, outperforming single-shot simulation by 40.07% and trained world-model baselines by 12.36% on benchmarks such as τ-bench, AppWorld, and OfficeBench. This approach provides a practical foundation for safe and scalable agentic test-time scaling in shared, non-resettable environments.

By Wendi Li, Huseyin A. Inan, Wei-Ning Chen, Yanzhi Chen, Chetan Bansal, Saravan Rajmohan, Sharon Li