Network-Wide Energy-Efficiency Maximization in UAV-Aided IoT Networks: Quasi-Distributed Deep Reinforcement Learning Approach - Lee, Seungmin; Ban, Tae Won; Lee, Howon (IEEE Internet of Things Journal, 2025)
In uncrewed aerial vehicle (UAV)-aided Internet of Things (IoT) networks, providing seamless and reliable wireless connectivity to ground devices (GDs) is difficult owing to the short battery lifetimes of UAVs. Hence, we consider a deep reinforcement learning (DRL)-based UAV base station (UAV-BS) control method to maximize the network-wide energy efficiency of UAV-aided IoT networks featuring continuously moving GDs. First, we introduce two centralized DRL approaches; round-robin deep Q-learning (RR-DQL) and selective-k deep Q-learning (SKDQL), where all UAV-BSs are controlled by a ground control station that collects the status information of UAV-BSs and determines their actions. However, significant signaling overhead and undesired processing latency can occur in these centralized approaches. Hence, we herein propose a quasi-distributed DQLbased UAV-BS control (QD-DQL) method that determines the actions of each agent based on its local information. By performing intensive simulations, we verify the algorithmic robustness and performance excellence of the proposed QD-DQL method based on comparison with several benchmark methods (i.e., RRDQL, SK-DQL, multiagent Q-learning, and exhaustive search method) while considering the mobility of GDs and the increase in the number of UAV-BSs.
Keywords: Aerial vehicle; Efficiency maximization; Energy; Multi agent; Multi-agent deep reinforcement learning; Network-wide energy efficiency maximization; Reinforcement learnings; Unmanned aerial vehicle control; Unmanned aerial vehicle-aided internet of thing network; Unmanned aerial vehicle-base station; Vehicle Control; Multiagent deep reinforcement learning (DRL); network-wide energy efficiency maximization; UAV Control; UAV-base station (BS); uncrewed aerial vehicle (UAV)-aided Internet of Things (IoT) network