Markov Decision Policies for Distributed Angular Routing in LEO Mobile Satellite Constellation Networks

Abstract

This article proposes a distributed angular routing algorithm in time-varying dynamic low Earth orbit (LEO) satellite constellation networks. For designing satellite routing algorithms, it is essential to consider 1) distributed operation due to the difficulty in global centralized computation and 2) angle-based computation under the consideration of orbit coordinate systems. Therefore, our proposed routing algorithm is based on distributed angular computation. Moreover, the proposed algorithm is designed by the Markov decision process (MDP) for discrete-time sequential decision making in time-varying LEO satellite networks. As a result, this article proposes an MDP-based distributed angular routing (MDAR) algorithm for seamless LEO routing. Based on the reward formulation in terms of angular differences in MDP formulation, our proposed distributed angular routing algorithm pursues orbit-geometrically straight-line data delivery from the source to its associated destination. Finally, our proposed routing algorithm is evaluated in the realistic environment with real-world satellite data, i.e., two line elements (TLEs), and the results confirm that our proposed algorithm outperforms the others in terms of routing success rate, reward convergence, and successful throughput.