Autonomous Drone Control: A Reinforcement Learning Approach

Abstract

Drones have become an essential tools across a wide range of industries, from agri- culture to surveillance, and are increasingly deployed in military contexts for detection, recognition, identification, exploration, and combat purposes. While most systems remain controlled by human, the shift toward autonomy is intensifying, driven by breakthroughs in artificial intelligence, notably in reinforcement learning and scalable simulation tech- niques. This Master’s thesis explores the potential of reinforcement learning for drone control within both single-agent and multi-agent frameworks. Two tasks are addressed : naviga- tion in unknown terrains and adversarial drone combat. Our work focuses on designing simulation environments that model the learning process of agents as they interact with these tasks. Our navigation environment consists of multiple randomly spaced obstacles (spikes), a target, and a drone placed on opposite sides of the terrain. The drone is equip- ped with a sensor—either a LiDAR or a camera—which it uses to explore the environment and reach the target. In the adversarial scenario, the environment includes two drones : an attacker and a defender. The attacker attempts to reach a designated target, while the defender tries to intercept it by colliding with it. Reinforcement learning is particularly well suited to these tasks due to its ability to learn complex, sequential decision-making policies from interaction with the environment. In scenarios such as drone navigation or combat, where the environment is often partially observable, highly dynamic, and difficult to model analytically, RL offers a flexible and data-driven approach to learning effective control strategies. Furthermore, Reinforcement learning naturally supports learning in multi-agent settings, where agents must coordinate or compete in real time. To tackle these tasks, policy gradient methods such as Proximal Policy Optimization , its multi-agent extension Independent Proximal Policy Optimization and a variant ins- pired by self-play methods were explored. To train and evaluate our agents, IsaacLab environments were designed following the formalism of partially observable Markov deci- sion process and stochastic games. Our work highlights the performance of trained agents regarding these tasks and show promising potential for future improvements regarding autonomous drone control