UAV Pilot Status Identification Algorithm Using Image Recognition and Biosignals

Abstract

With the development of various technologies such as sensors and communications, the scope of application for UAVs (unmanned aerial vehicles) is expanding. The use of UAVs is increasing not only in the military sectors, but also in the civilian industries. For the operation of UAVs, pilots must use a control system called the GCS (ground control system). With the GCS, pilots need to understand and be informed of the full operational context of the UAVs. However, the GCS can only provide the pilots with limited resources. Therefore, in order to overcome these limitations, excessive information may be provided to the pilots, which may cause abnormal conditions such as mission overload. In this context, there is a need for a system that can prevent abnormal conditions of the pilot and increase the mission success rate. In this paper, the pilot state information is collected through a camera and wearable devices to understand the pilot state in real time. An algorithm that can derive the pilot state from the collected information was developed. Algorithms can provide feedback to prevent accidents caused by mistakes and contingencies that can arise from the pilot's abnormal conditions. The algorithm shows high accuracy and stability when applied to simulated flight conditions. In addition, it is simple to use and there are no physical restrictions on the pilot's action, hence efficient mission performance is expected.