The paper introduces an explicit feedback contract for iterative machine‑learning agents, limiting when agents can observe validation scores and when they may stop iterating. The contract only releases a score once it exceeds a noise threshold derived from the standard error, preventing adaptive over‑selection of the validation split. Experiments on click‑through‑rate prediction, learning‑to‑rank, and a zero‑signal control task show that the contract reduces the validation‑test gap, yields slightly higher test performance, and forces agents to stop earlier—using only 4–12 submissions instead of the full 30 budget used with exact feedback.
arXiv:2607. 25152v1 Announce Type: new Abstract: Long-running autonomous agents plan, act, and judge their own completion without human intervention.
LoopArena is a benchmark that evaluates how effectively a model (the Controller) can guide a separate coding agent (the Worker) through long-running tasks by monitoring progress, assigning work, and deciding when to stop. It consists of three settings: Type I with 160 low‑cost control questions, Type II with 80 task slices, and Type III with 80 full tasks. In full tasks, the best observed strict success rate is 36.25%, and Controllers can reduce inference cost by an average of 60.6% while maintaining a strong correlation in performance ranking.
arXiv:2607. 14408v1 Announce Type: new Abstract: A self-evolving agentic loop repeatedly proposes a tweaked version of an agent (its prompt template or program) and accepts or rejects the change based on a per-iteration quality signal.
The paper investigates whether an agent’s own verifier remains reliable when the agent is optimized against it. It introduces a stress‑testing framework that measures the pressure ratio ρ, the influence of the verifier on the agent’s search, and four falsifiable probes targeting common failure modes. Applying this framework to the Rationale Consistency Score (RCS) in an LLM agent that designs machine‑learning pipelines, the authors find that the verifier’s impact is minimal in their baseline setup, that agents can shorten claim lists without affecting search, and that pre‑commit probing can hide declines in faithfulness. The study concludes with a proposed reporting standard to audit verifiers before trusting them.
JevAgentHarness is a new framework that benchmarks, deploys, and trains decision models to share control with large language models (LLMs) in agentic tasks. It introduces AgentDecisionBench to evaluate decision models from isolated questions to full agent control, revealing that accurate isolated decisions still miss many tasks and that naïve delegation can reduce success. By fitting confidence thresholds per choice and training a 4B decision model on states after agent mistakes, JevAgentHarness achieves high task success while cutting cost and speeding up execution, with results and resources made publicly available.
arXiv:2608. 02636v1 Announce Type: cross Abstract: Self-evolving skill systems promise to improve agents by turning execution feedback into persistent skill updates without changing the underlying model.
Large language model (LLM) agents now fix software, operate computers and serve customers, but their harnesses, the programs that run them, call a large LLM at every step, which makes long tasks slow...
CAFE (Coupled Agent–Feedback Evolution) is a framework that lets a shared‑parameter model alternate between acting as a search agent and as a critic that provides corrective feedback. By learning when to request feedback and how to use it, CAFE trains the agent to recover from its own failures and shapes rewards both online and offline. Experiments on seven search benchmarks show that CAFE outperforms other RL‑based agents, maintains gains on out‑of‑domain tests, and reduces hallucinations, indicating that co‑evolving feedback is essential for self‑improving search agents.
arXiv:2603.01209v3 Announce Type: replace Abstract: In CodeAct, language-model agents write Python that calls tools and use execution feedback to choose actions. Persistent runtimes preserve Python v...
SkillGate is a method that trains agents to select the correct skill from a large slate during an episode by separating credit signals for skill selection and execution. It addresses the problem of selector credit starvation, where traditional outcome-rewarded RL fails to give sufficient credit to the skill-naming tokens, especially in long-horizon tasks. Experiments on five benchmarks show that SkillGate improves a 9B policy’s success rate from 40.8% to 53.2%, reduces exposure to misleading candidates, and requires fewer skill reads.
Opera is a verbal critic framework designed for long‑horizon coding agents, treating each correction as a persistent note that is followed until the problem is resolved. It uses periodic and event‑driven triggers to diagnose issues, audits feedback against visible evidence, and tracks subsequent actions to distinguish compliance from true resolution. In tests, Opera improves the resolve rate of non‑critic agents by up to 12.4, 15.0, and 8.9 percentage points on three benchmarks and provides on‑policy training data that boosts a model’s performance without requiring a critic at inference time.
arXiv:2609.25237v1 Announce Type: new Abstract: Post-training is becoming a service (PTaaS): a customer hands an operator data and a goal, and a forward-deployed engineer (FDE) returns a fine-tuned,...