Sandman is a sandbox service developed at Amazon for executing and evaluating agentic reinforcement learning (RL) tools on shared Kubernetes clusters. It creates lightweight Kubernetes pods that run a small daemon inside unmodified task images, allowing files and processes to persist across repeated tool calls until the training framework releases the environment. In large-scale tests, Sandman supported 11,858 concurrent tiny sandboxes and handled 3.1 million sandboxes in a single month, demonstrating its ability to manage creation bursts, environment preparation, and cleanup for production RL workloads.
arXiv:2602. 11210v5 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has become a key paradigm for training software engineering (SWE) agents, but existing pipelines typically rely on per-task containers for isolation.
arXiv:2607. 01415v1 Announce Type: new Abstract: Coding-agent reinforcement learning treats execution infrastructure as a background implementation detail, despite relying on large numbers of interactive software rollouts.
arXiv:2602. 09345v3 Announce Type: replace-cross Abstract: AI agents are increasingly deployed in multi-tenant cloud environments, where they execute diverse tool calls within sandboxed containers, each call with distinct resource demands and rapid fluctuations.
Agentic‑Kube is a cooperative multi‑agent reinforcement learning framework for Kubernetes pod placement that splits the multi‑objective scheduling problem into cost minimisation, anti‑affinity fault tolerance, and vector resource balancing, each handled by a dedicated sub‑agent. It uses a bipartite Graph Convolutional Network to model host‑pod dependencies, a two‑stage monotonic QMIX value factorisation network for joint action coherence, and a plurality voting consensus with action feasibility masking. Evaluations on Google Kubernetes Engine and large‑scale clusters show Pareto‑efficient placements, a 53% reduction in anti‑affinity collisions, a 65% spot instance allocation ratio, and sub‑30 ms decision latencies up to 1,000 nodes without container restarts.
AgentFly is an agentic reinforcement learning framework that introduces a unified resource layer to manage heterogeneous environments—such as sandboxes, model services, and external APIs—through a single scheduling engine. The framework is organized into four layers: an agent layer that abstracts agents, tools, and rewards; a rollout layer that composes agent loops and computes rewards; a context layer that organizes rollouts and injects contextual information; and a low‑level resource layer that handles resource management. AgentFly offers prebuilt tools and environments, demonstrates successful agent training across multiple tasks and models, and provides the first controlled cross‑framework throughput comparison against other agentic RL frameworks.
arXiv:2607. 21557v1 Announce Type: new Abstract: Modern AI agents rely on elaborate inference harnesses such as Claude Code, Codex, and OpenClaw to drive multi-turn reasoning, tool use, and access to external systems.
AgentMercury is a scalable framework that synthesizes executable environments from high‑level business scenarios instead of task‑specific benchmarks. It creates a persistent world with entities, services, tools, and invariants, allowing diverse tasks and interaction trajectories to emerge naturally. The authors generated 4,783 environments across 14 industries and 50 countries, and training reinforcement‑learning agents on them improved performance on enterprise workflows and out‑of‑domain benchmarks, while the construction process itself can be learned to increase authoring success.
arXiv:2601. 07376v2 Announce Type: replace Abstract: We introduce \textsc{OpenTinker}, an open infrastructure for training large language model (LLM) agents with many LoRA-backed policies over shared execution resources.
arXiv:2608.22167v1 Announce Type: new Abstract: Reinforcement learning (RL) has become an effective way to improve the tool-use ability of large language models (LLMs), but most existing RL framework...
arXiv:2605. 22781v2 Announce Type: replace-cross Abstract: LLM-powered AI agents require high-frequency state exploration (e.
LEGO-RL is a framework that connects native coding-agent harnesses with scalable policy‑gradient training without altering the harnesses’ internal flow. It achieves faithful optimization through in‑process LLM proxying, reliable execution via sandbox orchestration, and observable training with automated validation and a Live UI. Experiments show LEGO‑RL improves the Qwen3.5‑35B‑A3B model’s performance on three native harnesses while preserving high rollout‑training probability correlation.
arXiv:2605. 25160v2 Announce Type: replace Abstract: GUI agents powered by large language models are advancing rapidly, creating urgent needs for evaluation and training based on realistic environments.