AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,733 stories · RSS feed

arXiv Computer Vision
Oct 15

Player-1: Learning to Play from Synthetic Experience

arXiv:2610.20584v1 Announce Type: new Abstract: Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions. However, games, with their...

By Mingyu Ouyang, Yiran Wang, Siyuan Hu, Xingyilang Yin, Huiqiang Sun, Guangzhi Wang, Junfu Pu, Cheng Cheng, Honghao Chen, Ying Shan, Mike Zheng Shou, Hwee Tou Ng
arXiv Machine Learning
Oct 15

L3M: A Verifiable Harness and Runtime for Self-Extending and Self-Improving Agents

L3M is a runtime built in Lean 4 that enables self‑extending and self‑improving agents to operate across humans, models, and code without requiring human attention in every loop. It stores all computation state in Git, allowing rich state trajectories that can be resumed, rewound, forked, compared, and merged. The system uses reversibility and kernel‑checked theorems to ensure safety, providing agents with sealed Lean programs as tools that can be reviewed and undone when necessary.

By Carson Eisenach, Robert Joseph George, Vincent Quenneville-Belair, Kate Gwimm, Dean Foster
arXiv Machine Learning
Oct 15

StateCraft: Augmenting Video World Models with Abstract-State Reasoning

StateCraft introduces a new architecture that separates a persistent, updatable abstract state from the visual state in video world models. By pairing a language model with a video diffusion model, it tracks entity changes over time and conditions visual generation on this abstract state. The approach is evaluated on generative interfaces and a new benchmark, showing improved generation quality, memory usage, and controllability, while enabling direct editing of unseen states and extending to spatial 3D environments.

By Lorenzo Venturoli, Nedko Savov, Xi Wang, Marc Pollefeys, Luc Van Gool
arXiv Machine Learning
Oct 15

Calibration and Risk-Budgeted Decisions with System One Models

The paper investigates how large language model (LLM) workflows and agents make binary decisions—such as allowing or blocking an input—when human deferral is not possible. It proposes using System One decision scores that default to a conservative fallback in uncertain cases and control false omission risk within the subset of inputs that receive the utility‑maximizing action. Experiments on six datasets spanning moderation, safety, and reranking show that naive threshold baselines often exceed target risk, whereas a Learn‑Then‑Test approach achieves only 7.8% risk, demonstrating the limitations of probability‑like interfaces and the benefits of risk‑budgeted selection.

By Soundouss Messoudi, Youness Mansar
arXiv Machine Learning
Oct 15

Investigating Behavior Cloning from Few Demonstrations for Autonomous Driving based on Bird's-Eye View in Simulated Cities

The paper explores Behavior Cloning (BC) for autonomous driving using bird's‑eye view (BEV) inputs in simulated urban settings. It introduces a weighted BC approach that employs a kernel density estimator to adjust training sample weights, emphasizing rare but critical actions like stopping at red lights and accelerating at green lights. Experiments in the CARLA simulator show that this weighted method improves route completion and generalizes better to unseen test environments compared to standard BC.

By Eric Aislan Antonelo, Gustavo Claudio Karl Couto, Christian M\"oller, Pedro Henrique Fernandes
arXiv Machine Learning
Oct 15

MMLA-RSI: Recursive Self-Improvement through RTT and RTMA

The paper introduces MMLA‑RSI, a recursive self‑improvement framework within a Memory‑Mediated Learning Architecture. It presents two concrete implementations: Reasoning‑Time Training (RTT), which updates the policy state, and Reasoning‑Time Memory Adaptation (RTMA), which modifies the authoritative memory. Both routes treat their own adaptation mechanisms as improvable objects, enabling successive generations of agents to refine policy and memory through bounded rule languages and shared interfaces.

By Junyi Zou, Avrova Donz
arXiv Machine Learning
Oct 15

Hierarchical Residual Policy Learning for Real-World Mobile Manipulation with Sparse Human Guidance

The paper introduces HiRe-MoMa, a hierarchical residual policy learning framework that fine‑tunes a frozen base policy for mobile manipulation. It uses a policy‑agnostic residual split into mobile‑base and manipulator‑arm heads to reduce gradient interference, and incorporates expert‑guided value alignment to improve sample efficiency with sparse human interventions. Experiments on the TIAGo++ platform show that HiRe‑MoMa boosts task performance after only about 30 minutes of online interaction, regardless of the underlying base policy.

By Zhefei Gong, Hefeifei Jiang, Zehao Wang, Jiachen Li
arXiv Machine Learning
Oct 15

A Byzantine-Robust Federated Learning with Nesterov Momentum

The paper introduces Byrd-NAFL, a Byzantine‑robust federated learning algorithm that incorporates Nesterov momentum at the server while keeping workers stateless. It applies any soft γ‑Byzantine resilient aggregation rule and provides finite‑time convergence guarantees for smooth non‑convex losses across all momentum factors β∈[0,1). Experiments on MNIST, CIFAR‑10, and Sentiment140 demonstrate faster convergence and higher accuracy than existing benchmarks under various attacks.

By Lihan Xu, Xiaoyi Fan, Gang Wang, Runhao Zeng, Xiping Hu, Yanjie Dong
arXiv AI
Oct 15

LatentRemedy: Efficient Remedy For Non-Prefix KV Cache Reuse in Multi-Agent Systems

LatentRemedy is a method designed to improve inference efficiency in multi‑agent systems that share non‑prefix KV caches. By inserting a small number of task‑conditioned latent tokens generated via an MLP, the target agent can better interpret reused context without recomputing the shared block. Experiments demonstrate that LatentRemedy consistently outperforms existing reuse baselines on conflicting tasks while keeping TTFT close to direct reuse as cache size grows.

By Zhiwei Chen, Zhongtao Rao, Waikit Xiu, Yuxiang Zhang, Shuting Cui, Jian Chen, Tianxiang Zhao
arXiv AI
Oct 15

Cost-Efficient Privacy Enforcement for Email-Generating LLM Agents under Strategic Information Requests

The paper proposes a cost‑efficient privacy enforcement method for email‑generating large language model (LLM) agents that intercepts drafts, creates an information‑release contract, rewrites the draft locally with a Gemma rewriter, and then optionally calls a final PrivacyChecker. Experiments on normal and strategically malicious prompts show that the Gemma + Checker approach reduces leakage from 11.41 % to 4.03 % and improves usable output rates, while also cutting the cost amplification of checker‑only pipelines from 1.32× to 1.02×. The results demonstrate that moving repair to a cheaper local layer can significantly lower both privacy leakage and operational cost, yet still rely on a checker for high‑risk sends.

By Qikun Cai, Tao Huang, Chen Hou