Master's Thesis : Coordination on the battlefield by multi-agent reinforcement learning.

Abstract

This study investigates Multi-Agent Reinforcement Learning algorithms that improve agent cooperation and coordination through agent-to-agent communication. The basics of Reinforcement Learning are presented after introducing the IRIS project and the high-level objectives of this work. Multi-Agent Reinforcement Learning's State-of-the-Art methods are then presented with a focus on algorithms which allow agents to learn to communicate. An algorithm designed to allow use different methods to aggregate messages and that is able to learn when to send message is presented. Moreover, the possibility to leverage Multi-Agent critics is shown. Within the Starcraft II Multi-Agent Challenge, variations of the communicating algorithm are tested on a hand-crafted scenario. Various performance and behavior metrics are analyzed and compared with a non-communicating algorithm. Results show that, although final performance is similar, allowing agents to communicate increases the learning rate and leads to behaviorally different agents.