The paper investigates code-level autonomous research loops (ARLs) where a language model edits training pipelines to improve an in-loop metric. It identifies a failure mode called algorithmic mode collapse, where edits become semantically uniform despite surface diversity, leading to a growing gap between in-loop gains and independent evaluation. The authors propose Diversity‑Aware Proposal Sampling (DAPS), a lightweight method that reduces semantic decay by 69.1% and boosts faithfulness by over 80% while maintaining optimization speed.
arXiv:2608. 05179v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly used across the scientific research lifecycle: ideation, literature search, experiment design and execution, analysis, manuscript drafting, and review.
arXiv:2606. 05646v1 Announce Type: cross Abstract: Large language models (LLMs) have enabled powerful software engineering (SE) agents capable of navigating complex codebases and resolving real-world issues.
TestJack is a scalable framework that audits large language model (LLM) coding agents by generating new tests for each trial, focusing on prompt requirements that may be violated. It keeps only the tests passed by the ground‑truth patch and re‑examines failures, providing replayable evidence for each confirmed error. In experiments across six model backends and five benchmarks, TestJack found that 34.4% of trials deemed correct actually violated task requirements, reducing the overall resolution rate from 50.6% to 33.2%.
arXiv:2604.16625v2 Announce Type: replace-cross Abstract: Recent large language model (LLM) agents have shown promise in using execution feedback for test-time adaptation. However, robust self-improv...
AgentFold is a multi‑agent framework that treats protein‑folding model design as a closed‑loop search over executable code variants. Starting from the ESMFold codebase, the agents generate hypotheses, modify and debug code, evaluate model variants, and store both successes and failures in structured memory, guided by an MCTS‑style policy that allocates GPU resources. In an engineering‑scale experiment, AgentFold explored about 80 variants using 5,000 GPU‑hours and 170 million LLM tokens, improving the best lDDT score by 7.5% over independent Codex proposals and outperforming a random‑search baseline, while also uncovering empirical design patterns such as the benefits of early, soft, learnable priors.
SAGE-Loop is a new closed‑loop, self‑adaptive AutoML framework that uses large language models to generate and validate machine learning pipelines in multiple rounds, allowing trial‑and‑repair and adaptive ensemble selection for both supervised and unsupervised tasks. It addresses the lack of instant feedback and correction in existing AutoML by enabling process‑level recovery from failures and dynamic use of model diversity. Experiments on 20 public datasets show consistent improvements in performance and stability across classification, regression, and clustering, and demonstrate the system’s ability to recover from execution failures.
RankEvolve is an auto‑research framework that evolves generative ranking models by orchestrating multiple large‑language‑model coding agents through an Executable Operating Protocol (EOP). The system compiles a state machine that enforces phases, gates, branches, and loops, while a meta‑meta‑harness lets agents review and repair each other’s code. In budget‑matched experiments, heterogeneous composition of agents raised execution accuracy from 45.8 % to 62.5 % and reduced silent critical‑defect rates, achieving notable gains on the HSTU recommender and other benchmarks.
arXiv:2607. 18161v1 Announce Type: cross Abstract: Coding agents are increasingly used to accelerate code generation in many downstream tasks, such as fixing bugs, building applications, and prototyping.
The paper introduces a domain‑specific debugging agent for repairing near‑miss accelerator kernels, arguing that debugging is more efficient than regenerating kernels from scratch. It tackles knowledge scarcity, integrity, and cost control through pattern retrieval, anti‑cheat detection, and convergence safeguards. Empirical results show the debug agent achieves a 66.7% Pass@1 rate and uses 92.8% fewer tokens per success compared to traditional regeneration methods.
AI code agents are increasingly deployed to resolve real software issues, yet their reliability under superficial code variations remains poorly understood. We evaluate whether coding agents that repa...
The paper introduces CABRA, a synthetic benchmark that constructs coding tasks from scratch using call‑graph transformations and scales difficulty along four axes: function traversal, search, runtime resolution, and instruction following. Experiments with eight large language models and six coding agents on 6,840 CABRA tasks show that while LLM accuracy degrades with task size, agents maintain high performance by leveraging tools; larger tasks trigger more tool calls, and a separate study on SWE‑bench verifies that tool‑call counts better predict agent accuracy than lines of code edited. The authors argue that CABRA exposes weaknesses in LLMs that are hidden by tool use and highlights the importance of code‑understanding abilities beyond mere editing. whyItMatters":"The study demonstrates that coding agents’ performance is more dependent on code‑understanding than on the amount of code edited, revealing a critical bottleneck for current coding agents."
arXiv:2607. 20499v1 Announce Type: new Abstract: Large Language Models generate plausible backend code, but a single-pass paradigm provides no guarantee of correctness or runtime reliability.