← Back to all news
arXiv AI By Yiming Yan, Hongyang Wang, Wenbo Chen, Xuan Wang, Jingguo Liu, Xian Li, Chen Qi, Leo Fan, Siwei Li, Qianli Wu, Yichuan Li, Ruochen Wen, Qian Qu, Zheshen Wang, Benoit Dumoulin

Sandman: Building and Operating Sandboxes for Agentic Reinforcement Learning on Shared Kubernetes

Read the original on arXiv AI →

Sandman is a sandbox service developed at Amazon for executing and evaluating agentic reinforcement learning (RL) tools on shared Kubernetes clusters. It creates lightweight Kubernetes pods that run a small daemon inside unmodified task images, allowing files and processes to persist across repeated tool calls until the training framework releases the environment. In large-scale tests, Sandman supported 11,858 concurrent tiny sandboxes and handled 3.1 million sandboxes in a single month, demonstrating its ability to manage creation bursts, environment preparation, and cleanup for production RL workloads.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Aug 31

Agentic-Kube: A Graph-Enhanced Multi-Agent Reinforcement Learning Framework for Multi-Objective Kubernetes Scheduling

Agentic‑Kube is a cooperative multi‑agent reinforcement learning framework for Kubernetes pod placement that splits the multi‑objective scheduling problem into cost minimisation, anti‑affinity fault tolerance, and vector resource balancing, each handled by a dedicated sub‑agent. It uses a bipartite Graph Convolutional Network to model host‑pod dependencies, a two‑stage monotonic QMIX value factorisation network for joint action coherence, and a plurality voting consensus with action feasibility masking. Evaluations on Google Kubernetes Engine and large‑scale clusters show Pareto‑efficient placements, a 53% reduction in anti‑affinity collisions, a 65% spot instance allocation ratio, and sub‑30 ms decision latencies up to 1,000 nodes without container restarts.

By Hamed Hamzeh
agentsreinforcement-learningbenchmarks
More like this →
arXiv AI
Oct 9

AgentFly: Scaling Agentic Reinforcement Learning with Unified Resource System

AgentFly is an agentic reinforcement learning framework that introduces a unified resource layer to manage heterogeneous environments—such as sandboxes, model services, and external APIs—through a single scheduling engine. The framework is organized into four layers: an agent layer that abstracts agents, tools, and rewards; a rollout layer that composes agent loops and computes rewards; a context layer that organizes rollouts and injects contextual information; and a low‑level resource layer that handles resource management. AgentFly offers prebuilt tools and environments, demonstrates successful agent training across multiple tasks and models, and provides the first controlled cross‑framework throughput comparison against other agentic RL frameworks.

By Renxi Wang, Rifo Ahmad Genadi, Bilal El Bouardi, Yongxin Wang, Fajri Koto, Zhengzhong Liu, Timothy Baldwin, Haonan Li
llmsagentsreinforcement-learningfine-tuningefficiency
More like this →