arXiv AI

Hierarchical Reinforcement Learning in StarCraft Micromanagement with Influence Maps and Cluster-based Scripts

arXiv:2606. 30092v1 Announce Type: new Abstract: Real-time strategy (RTS) games present significant AI challenges, characterized by expansive state-action spaces arising from multi-unit coordination in continuous battlefields, and sparse delayed rewards stemming from final win/lose signals.

arXiv AI
Jun 30

SAT-RTS: A systematic framework for tactical knowledge extraction and visualization-based analysis in real-time strategy games

arXiv:2606. 30090v1 Announce Type: new Abstract: Efficient tactical knowledge extraction and analysis in real-time strategy (RTS) games micromanagement are constrained by the high-dimensional coupled state-action sequential data and the black-box decision-making process.

By Chunhui Bai, Changhe Li, Yuqiang Li, Lei Liu, Shoufei Han
arXiv Machine Learning
Aug 27

BVR Sim: An Open and High-Throughput Environment for Heterogeneous Air-Combat Reinforcement Learning

BVR Sim is an open‑source, Gymnasium‑style environment for heterogeneous air‑combat reinforcement learning, supporting multiple JSBSim aircraft models (F‑15, F‑16, F/A‑18, F‑22) with configurable weapons, sensors, and opponents. It offers a unified tactical action interface, interchangeable Python and accelerated C++ backends, entity‑oriented observations, compositional rewards, scripted opponents, replay and visualization, and adapters for multi‑agent learning frameworks. At a 0.4‑second decision interval, the C++ backend achieves 104 simulated seconds per wall‑clock second in 1‑vs‑1 and remains practical through 10‑vs‑10 scenarios, and a policy trained on the F‑16 transfers to four unseen aircraft with a 45.5% mean win rate after controller adaptation.

By Haocheng Sun (Beijing University of Posts,Telecommunications), Mulai Tan (Air Force Engineering University)
arXiv AI
Jun 19

Hierarchical Control in Multi-Agent Games: LLM-based Planning and RL Execution

arXiv:2606. 20014v1 Announce Type: cross Abstract: Reinforcement learning (RL) has achieved strong performance in sequential decision-making, yet scaling to complex multi-agent environments remains challenging due to sparse rewards, large state-action spaces, and the difficulty of learning coordinated strategies.

By Jannik H\"osch, Alessandro Sestini, Florian Fuchs, Amir Baghi, Joakim Bergdahl, Konrad Tollmar, Jean-Philippe Barrette-LaPierre, Linus Gissl\'en
arXiv AI
Sep 12

DRG-MAPPO: Hierarchical Dynamic Role-Graph Multi-Agent Reinforcement Learning for Cooperative Air Combat

The paper introduces DRG-MAPPO, a hierarchical multi‑agent reinforcement learning framework for cooperative air combat. It combines graph‑based relational modeling with dynamic role assignment, using a high‑level policy to allocate tactical roles such as leader and supporter, and a low‑level policy to execute maneuver actions. The approach includes a target‑priority auxiliary task and achieves an 87% win rate in experiments, indicating effective coordination and stability.

By Junlin Liu, Chengwei Li, Yang Gao, Hui Chang, Xinchen Zhang, Zhijun Zhao, Hao Zhao
arXiv Machine Learning
Sep 4

LLM-Guided Reinforcement Learning for Adaptive NPC Behavior in Multi-Agent Combat Games

The paper explores a runtime strategy-selection framework where a large language model (LLM) guides a pre‑trained reinforcement learning (RL) policy for non‑player characters (NPCs) in a Unity combat game without altering the underlying policy. Five NPC agents sharing a PPO policy were compared in a baseline setup and an LLM‑augmented setup, where a locally hosted Mistral 7B model assigns one of four tactical tags every five seconds based on live game state. Across 600 episodes against three scripted opponents, the LLM‑augmented agents more than doubled their win rate against a Balanced opponent, improved performance against an Evasive opponent, but struggled against an Aggressive opponent due to over‑reliance on encirclement; analysis of 2,430 strategy selections revealed limited zero‑shot differentiation with the model favoring Surround in 83.8% of cases.

By Hrithika Deepu Nair, Kayvan Karim
arXiv Machine Learning
Jul 21

Hierarchical Reinforcement Learning for Air Combat at DARPA's AlphaDogfight Trials

arXiv:2105. 00990v3 Announce Type: replace Abstract: Autonomous control in high-dimensional, continuous state spaces is a persistent and important challenge in the fields of robotics and artificial intelligence.

By Adrian P. Pope, Jaime S. Ide, Daria Micovic, Henry Diaz, David Rosenbluth, Lee Ritholtz, Jason C. Twedt, Thayne T. Walker, Kevin Alcedo, Daniel Javorsek
arXiv Machine Learning
Sep 15

Hierarchical Deep Counterfactual Regret Minimization

arXiv:2305.17327v4 Announce Type: replace Abstract: Imperfect Information Games (IIGs) are used to model games under uncertainty or lack complete information. Counterfactual Regret Minimization (CFR)...

By Jiayu Chen, Xudong Wu, Zhekai Wang, Vaneet Aggarwal