AutoFyn is a new agent framework based on the Expert Iteration algorithm that updates a frozen model’s persistent state using verified reward signals instead of changing model weights. Each iteration starts with a fresh model session, and only durable information—such as memory files, reports, and repository state—is carried over through explicit interfaces. The system orchestrates exploration, planning, and specialized agents, while a task‑grounded verifier supplies objective rewards that are distilled back into the persistent state to guide the next round. AutoFyn has been applied to olympiad mathematics, data science, and cybersecurity, outperforming baseline models on the 2026 International Mathematical Olympiad, topping the Spider 2.0 dbt benchmark, and generating sixteen maintainer‑confirmed vulnerability advisories across several popular software projects.
arXiv:2609.22792v1 Announce Type: cross Abstract: LLM agents are increasingly used for security tasks: vulnerability discovery, exploit reproduction, and patch generation. Improving them at the model...
arXiv:2606. 07412v1 Announce Type: cross Abstract: LLM-driven software engineering agents have become a central testbed for real-world language-model capability, yet their training remains limited by the availability of high-quality SWE tasks.
arXiv:2607. 05471v1 Announce Type: cross Abstract: We present KAT-Coder-V2.
The paper introduces Traverse, a benchmark of 2,518 agent trajectories and 6,967 annotated mistakes across software engineering, computer use, and science tasks, revealing that failures often go unrecovered and can cause irreversible harm before a run is deemed successful. It shows that human judges struggle to detect the first mistake in most runs, while a 4‑billion‑parameter verifier called Scout can locate failures more effectively and improve task success when used to select among candidate runs. The study demonstrates that making failure detection inexpensive and reliable can enable long‑horizon agents to learn from their own mistakes and increase trustworthiness in autonomous AI.
arXiv:2607. 20531v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly deployed over Model Context Protocol (MCP) servers, yet the benchmarks used to evaluate them score the final answer or a fixed "ground-truth" list of tools, both of which are fragile once the underlying data is live and stateful.
arXiv:2512. 18552v3 Announce Type: replace-cross Abstract: While current software agents powered by large language models (LLMs) and agentic reinforcement learning (RL) can boost programmer productivity, their training data (e.
The paper introduces neuro‑symbolic computer use, a method that learns reusable policies to execute recurring computer workflows efficiently. Instead of re‑planning each run, the learned policy encodes stable decisions (ordering, variables, loops, branches) into executable code while delegating observation‑dependent decisions to neural models. Using neuro‑symbolic policy iteration, the approach iteratively refines the policy from a single agent trajectory, diagnoses failures, and revises the code with a coding model, achieving superior Pass^3 scores and significant reductions in per‑run cost and latency on OSWorld‑Verified and ScienceBoard benchmarks.
arXiv:2606. 04455v1 Announce Type: new Abstract: Current AI benchmarks evaluate agents on task execution within human-designed workflows.
arXiv:2607. 04542v1 Announce Type: cross Abstract: Every LLM agent run re-derives its behavior token by token on a frontier model: brilliant, expensive, slow, and unbounded.
BekchiAI introduces a benchmark and platform for evaluating large language model agents. The benchmark comprises 13 tool‑using ReAct agents across seven task categories, totaling 2,057 deterministic test tasks with verifier‑checkable gold answers. The platform offers web‑based observability, token and latency telemetry, and remote run termination for live agents.
arXiv:2607. 01764v1 Announce Type: new Abstract: Repository-level vulnerability reproduction is a demanding software engineering (SE) task: an agent must inspect a codebase, infer the input grammar that reaches a vulnerable path, construct a proof-of-conceptv(PoC), and verify that the crash disappears on the patched build.
arXiv:2608. 06471v1 Announce Type: cross Abstract: Despite recent advances, frontier large language model (LLM) agents remain limited in discovering and patching complex vulnerabilities in real-world software.