Autonomous Drone Delivery Navigation via Reinforcement Learning

Abstract

Autonomous drone delivery is increasingly recognized for its potential to expedite and decarbonize lastmile logistics. However, challenges such as congested airspace, restricted flight zones, and limited onboard energy persist. This dissertation presents a two-stage reinforcement-learning architecture designed to address these issues. In the first stage, the single-drone delivery problem is formulated as a Markov Decision Process (MDP), where time- and energy-efficient trajectories are learned using both tabular Q-learning and its Double Q-learning variant. Empirical results in a prototypical grid environment show that Double Q-learning accelerates convergence by approximately 30% and yields routes that are, on average, 15% shorter than those produced by standard Q-learning. The second stage focuses on collaborative multi-drone operations through MATAC (Multi-Agent Transformer-based Actor-Critic), enhanced with a Lagrangian proximal policy optimization scheme. Two separate critics are employed to maximize performance and strictly enforce no-fly zone constraints simultaneously. In large-scale urban simulations, MATAC achieves near-zero constraint violations, reduces average delivery time by 25%, and lowers per-drone energy consumption by 20% compared to capacity-matched MLP+PPO and unconstrained PPO baselines.