arXiv Machine Learning
Sep 4

Towards Scaling Reinforcement Learning to Massive Populations: Learning Mean-Field Representations

The paper proposes a mean‑field reinforcement learning framework that models rewards and transitions as functions of an unknown low‑dimensional aggregate statistic of a large agent population. By learning this low‑dimensional representation in an offline setting, the authors demonstrate a provable method for obtaining near‑optimal policies. Experiments on a one‑step routing game inspired by supply‑chain problems show that, with a fixed neural‑network size and optimization budget, the learned representation improves reward prediction and the quality of Nash equilibria compared to baselines that ignore population structure.

By Aditya Makkar, Benjamin Unger, Jeongyeol Kwon, Mathieu Lauri\`ere, Eugene Vinitsky, Yonathan Efroni
arXiv AI
3d ago

Learning to Incentivize Perfect-Knowledge Agents with Unobserved Rewards

The paper studies repeated principal-agent games in a multi-armed bandit setting where the principal offers arm-specific incentives, the agent chooses an arm to maximize its own reward plus incentive, and the principal observes only the chosen arm and its own reward. Because the principal cannot see the agent’s reward, traditional estimation fails, so the authors develop an estimator based on the sequence of incentives and chosen arms, analogous to online inverse optimization. Using this estimator, they design a policy that achieves low (square‑root) regret and validate the approach with simulations for personalized incentive design in long‑term statin treatment.

By Ilgin Dogan, Zuo-Jun Max Shen, Anil Aswani