arXiv Machine Learning
4d ago

Restless Bandits with Individual Penalty Constraints: Near-Optimal Indices and Deep Reinforcement Learning

This paper studies Restless Multi‑Armed Bandits with individual penalty constraints for dynamic wireless networks, allowing each arm to have distinct performance limits such as energy, activation, or age of information. It introduces the Penalty‑Optimal Whittle (POW) index, which depends only on an arm’s transition kernel and its constraints, making it computable offline and independent of system‑wide parameters. The authors prove the POW index policy is asymptotically optimal, present a deep reinforcement learning method to learn the index online, and show through simulations that it outperforms existing policies.

By Nida Zamir, I-Hong Hou
arXiv AI
Sep 2

Bandits in Prod: Hyperparameter Optimization at Inference Time

The paper introduces Online Hyperparameter Optimization (OHPO), framing it as an infinitely many‑armed bandit problem over mixed and conditional search spaces. It proposes the IMABO framework, which couples any bandit policy with any oracle for proposing new configurations, and presents IMOSS—a restart‑free anytime policy with provable regret bounds. Experiments show that IMABO, combined with practical oracles such as TPE, an incumbent‑mutation oracle, and a pretrained tabular foundation model, outperforms random search across a range of settings from classical ML models to LLM‑based agents.

By Louis Abraham, Tuan-Anh Nguyen, Nicolas Devatine