← Back to all news
arXiv Machine Learning By Kehui Yao, Chuanbo Hu, Jiao Chen, Ramin Giahi, Reza Yousefi Maragheh, Lalitesh Morishetti, Ziwei Hong, Yiyun Su, Jianpeng Xu, Evren Korpeoglu, Kannan Achan

An Explicit Feedback Contract for Iterative ML-Agent Harnesses

Read the original on arXiv Machine Learning →

The paper introduces an explicit feedback contract for iterative machine‑learning agents, limiting when agents can observe validation scores and when they may stop iterating. The contract only releases a score once it exceeds a noise threshold derived from the standard error, preventing adaptive over‑selection of the validation split. Experiments on click‑through‑rate prediction, learning‑to‑rank, and a zero‑signal control task show that the contract reduces the validation‑test gap, yields slightly higher test performance, and forces agents to stop earlier—using only 4–12 submissions instead of the full 30 budget used with exact feedback.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Oct 15

LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering

LoopArena is a benchmark that evaluates how effectively a model (the Controller) can guide a separate coding agent (the Worker) through long-running tasks by monitoring progress, assigning work, and deciding when to stop. It consists of three settings: Type I with 160 low‑cost control questions, Type II with 80 task slices, and Type III with 80 full tasks. In full tasks, the best observed strict success rate is 36.25%, and Controllers can reduce inference cost by an average of 60.6% while maintaining a strong correlation in performance ranking.

By Yi Wang, Haopeng Zhang, Chengxiang Huang, Rui Dai, Kaikui Liu, Piotr Koniusz, Xiangxiang Chu
agentsbenchmarks
More like this →
arXiv AI
3d ago

When Can You Trust an Agent's Own Verifier? Stress-Testing Counterfactual Rationale Verification Under Search Pressure and Oracle Access

The paper investigates whether an agent’s own verifier remains reliable when the agent is optimized against it. It introduces a stress‑testing framework that measures the pressure ratio ρ, the influence of the verifier on the agent’s search, and four falsifiable probes targeting common failure modes. Applying this framework to the Rationale Consistency Score (RCS) in an LLM agent that designs machine‑learning pipelines, the authors find that the verifier’s impact is minimal in their baseline setup, that agents can shorten claim lists without affecting search, and that pre‑commit probing can hide declines in faithfulness. The study concludes with a proposed reporting standard to audit verifiers before trusting them.

By Jonghyun Lee, Moonsu Kim, Sunghwan Park
llmsagents
More like this →
arXiv AI
6d ago

JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs

JevAgentHarness is a new framework that benchmarks, deploys, and trains decision models to share control with large language models (LLMs) in agentic tasks. It introduces AgentDecisionBench to evaluate decision models from isolated questions to full agent control, revealing that accurate isolated decisions still miss many tasks and that naïve delegation can reduce success. By fitting confidence thresholds per choice and training a 4B decision model on states after agent mistakes, JevAgentHarness achieves high task success while cutting cost and speeding up execution, with results and resources made publicly available.

By Kunlun Zhu, Tao Feng, Xu Han, Heng Wang, Ling Yang, Xiangru Tang, Jiaxuan You, Heng Ji
llmsagentsbenchmarks
More like this →