MLE-World is a world model fine‑tuned from Qwen3.5‑4B that predicts the terminal output of a machine learning program—such as error traces, training logs, and validation scores—directly from its code and data description, without executing the program. It also provides a confidence score indicating the likelihood that the prediction is correct. Using these predictions, the authors design five test‑time‑scaling policies that compare multiple candidate programs per step, executing only those with low confidence, and demonstrate that MLE‑World matches real outputs on 77.9% of held‑out programs and improves agent performance by up to 22.1% with fewer real executions.
By Brian Ko, Han Wang, Hejia Geng, Li Erran Li, Sherry Yang
Large language model (LLM) agents now fix software, operate computers and serve customers, but their harnesses, the programs that run them, call a large LLM at every step, which makes long tasks slow...
arXiv:2607. 00164v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards can in principle train calibrated probabilistic forecasters, since a proper scoring rule such as the Brier score is computed from outcomes alone and is minimized in expectation by the true probability.
By Sadanand Singh, Allam Reddy, Manan Chopra
StoreBench is a live‑commerce environment where an autonomous agent runs a mid‑size online apparel store on a production‑grade backend, facing continuous customer orders, supplier price changes, and market shocks. The agent uses the same 29 merchant tools as a human operator, with a windowed operation budget that ties simulated time to actions, ensuring that model latency does not affect the simulation. Seven large language models were evaluated across 11 scenarios of 30–45 days and a full simulated year, but none matched the scripted smart‑triage policy, and human experts outperformed all models.
By Daksh Raghuvanshi, Ved Vedere, Yifan Wang
arXiv:2610.18760v1 Announce Type: new
Abstract: Test-time scaling for large language models (LLMs) can involve generating multiple candidate solutions for selection or revising an earlier attempt. To...
By Zheye Deng, Sha Li, Changlong Yu, Xin Liu, Qin Lu, Priyanka Nigam, Yangqiu Song
arXiv:2607. 05904v1 Announce Type: new Abstract: Training a language model against its own reference-free judgments (the premise of self-rewarding, self-play, and LLM-as-a-judge pipelines) assumes a model's verdict on a shown answer tracks correctness.
By Chenyu Zhou