arXiv AI
Jun 4
The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development?
arXiv:2606. 04455v1 Announce Type: new Abstract: Current AI benchmarks evaluate agents on task execution within human-designed workflows.
SelfOp: An Optimization Algorithm for Self-Improving Security Agents
Related stories
arXiv AI
Jul 3
Mastermind: Strategy-grounded Learning for Repository-Scale Vulnerability Reproduction
arXiv:2607. 01764v1 Announce Type: new Abstract: Repository-level vulnerability reproduction is a demanding software engineering (SE) task: an agent must inspect a codebase, infer the input grammar that reaches a vulnerable path, construct a proof-of-conceptv(PoC), and verify that the crash disappears on the patched build.
arXiv Computation and Language
Aug 25
CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild
arXiv:2608.23181v1 Announce Type: cross Abstract: As large language models (LLMs) continue to advance in coding capabilities, their potential in cybersecurity has drawn increasing research attention,...
arXiv AI
2d ago
Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer
The paper introduces a self‑evolving harness framework where a frozen language‑model agent first solves tasks and then edits its own harness based on run records. Using a 49‑line seed harness, the evolved harness improves average scores on in‑distribution benchmarks by 4.48 points and on out‑of‑distribution benchmarks by 12.64 points, surpassing Codex on the former and matching it on the latter. Continued evolution on a specific out‑of‑distribution benchmark further raises performance, and the study analyzes emergent mechanisms such as output truncation and history compaction.
arXiv AI
Aug 18
Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents
arXiv:2608. 15071v1 Announce Type: new Abstract: Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.
arXiv AI
Sep 2
HarnessEvolve: Learning from Reference Trajectories for Reliable Agent Self-Evolution
HarnessEvolve is a self‑evolving framework that improves agent harnesses—prompts, skills, tools, and execution logic—by learning from reference trajectories. It separates execution, evaluation, optimization, and gating into independent modules, addressing credit assignment failure, shortcut learning, and catastrophic forgetting. The approach uses reference trajectories to extract error signals, applies quality and performance gates to candidate updates, and validates updates on held‑out data, consistently outperforming state‑of‑the‑art baselines across diverse benchmarks.
arXiv AI
Aug 10
CyberForge: Verified Vulnerability Injection at Repository Level for Cybersecurity Agent Training
arXiv:2608. 06471v1 Announce Type: cross Abstract: Despite recent advances, frontier large language model (LLM) agents remain limited in discovering and patching complex vulnerabilities in real-world software.
arXiv AI
Jun 3
Toward Training Superintelligent Software Agents through Self-Play SWE-RL
arXiv:2512. 18552v3 Announce Type: replace-cross Abstract: While current software agents powered by large language models (LLMs) and agentic reinforcement learning (RL) can boost programmer productivity, their training data (e.
arXiv AI
Jun 16
Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment
arXiv:2606. 15441v1 Announce Type: cross Abstract: Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution.
arXiv AI
Aug 19
TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation
TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It evaluates candidate skills against source evidence and nine safety properties, then tests them in a controlled environment to capture execution traces and identify failures. The system iteratively refines skills based on these results, achieving high precision in vulnerability detection and significantly improving task performance and security rates.
arXiv AI
Jun 19
The Autonomy Tax: Defense Training Breaks LLM Agents
arXiv:2603. 19423v2 Announce Type: replace-cross Abstract: Large language model (LLM) agents increasingly rely on external tools (file operations, API calls, database transactions) to autonomously complete complex multi-step tasks.
arXiv AI
Jun 11
Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference
arXiv:2606. 05922v2 Announce Type: replace Abstract: AI agents rely on a harness of skills, tools, and workflows to solve complex problems.