Drone-based mixed reality: enhancing visualization for large-scale outdoor simulations with dynamic viewpoint adaptation using vision-based pose estimation methods

Abstract

In recent years, there has been a growing interest in the integration of drones across diverse sectors, particularly within architecture, engineering, and construction (AEC). The amalgamation of drones with mixed reality (MR) stands out as a promising avenue. Proposed applications include the comparison of design and actual objects, as well as landscape simulation in urban design. A previous study successfully developed a drone-viewpoint MR system with low model dependence, leveraging general drones and methods. However, the alignment between the real and virtual worlds was contingent on predefined flight routes, limiting adaptability during MR execution. This study introduces a new model-independent drone viewpoint MR system that integrates two vision-based attitude estimation methods, enabling execution on arbitrary flight paths. Evaluation of the prototype for system latency and alignment accuracy revealed an overall latency of 3.5 s. The alignment accuracy, assessed using intersection over union, demonstrated performance equal to or surpassing the previous system. While this paper does not showcase MR content for practical use, the research lays the groundwork for advancing drone applications in the AEC field. The proposed system offers versatility for MR applications across various stages, from design to maintenance.