Assessing Human Factors for Drone Operations in a Simulation Environment

Abstract

The number of small drones and drone operations is expanding and proliferating tremendously. However, there is a problem. Drones crash. Often. When they do, international studies show that over 70% of the drone crashes can be related to human factors. Combining these two facts, it is clear that - if we want to avoid a massive number of drone incidents in the future – it is required to develop a strategy to incorporate human factors in the drone deployment process and the training of drone pilots. Pilots for regular aircraft or for larger (typically military) drones generally follow extensive simulator training before engaging in any real flight. However, for small rotorcraft, this is much less the case, because it is very difficult to convey a realistic representation to the human sensory system. Both for fixed wing and rotary wing drones, the main problem with current simulator-based pilot training programs is that they are limited to simplistic scenarios (typically flying predefined patterns and practicing take-off and landing operations), without providing much qualitative feedback to the trainee or the supervising entity.In response to these identified shortcomings, we present in this paper a drone operator performance assessment tool, which uses a realistic environment and realistic operational conditions to measure the performance of the drone operator, both in a qualitative and quantitative manner. These metrics can then be used by training responsibles to adapt / adjust the theoretical and practical training courses for drone pilots, such that the curriculum (both the practical and the theoretical courses) can be iteratively optimized to best fit the needs. An important aspect of any qualification assessment procedure is the definition of the test methodologies and of the test scenarios. Within the subject of drone pilot training, these test scenarios are currently most often very limited to simple take-off & landing operations and of following simple patterns in the air. For pilots working in the security sector (military, police, firefighters, civil protection, ...) in tough operating conditions, these highly simplistic scenarios are hardly relevant. Therefore, we also propose a set of standard test methods specifically geared towards the training of drone operators in the security sector.In this paper, we present the architecture of the developed assessment tool, which runs inside a simulation environment, enabling repetitive (statistically relevant) testing in a controlled environment. The simulation engine is based upon the AirSim simulator and uses the UnReal game engine for realistic environmental rendering. Together with the end-users, twenty-two scenarios have been defined within two main mission environments: a mountainous and an urban environment. These scenarios are especially conceived to cover most of the hazards and environmental challenges that end users wanted to test for, the pilot performance capabilities that they wanted to see measured and the human factors that they identified as potentially important influencing factors for pilot performance.