Enhancing Maritime Navigation with Mixed Reality: Assessing Remote Pilotage Concepts and Technologies by In Situ Testing

Abstract

In response to the evolving landscape of maritime operations, new technologies are on the horizon as mixed reality (MR), which shall enhance navigation safety and efficiency during remote assistance as, e.g., in the remote pilotage use case. However, up to now, it is uncertain if this technology can provide benefits in terms of usability and situational awareness (SA) compared with screen-based visualizations, which are established in maritime navigation. Thus, this paper initially tests and assesses novel approaches to pilotage in the congested maritime environment, which integrates augmented reality (AR) for ship captains and virtual reality (VR) and desktop applications for pilots. The tested prototype employs AR glasses, notably the Hololens 2, to superimpose the Automatic Identification System (AIS) data directly into the captain’s field of view, while pilots on land receive identical information alongside live 360-degree video feeds from cameras installed on the ship. Additional minimum functionalities include waypoint setting, bearing indicators, and voice communication. The efficiency and usability of these technologies are evaluated through in situ tests conducted with experienced pilots on a real ship using the System Usability Scale, the Situational Awareness Rating Technique, as well as Simulator Sickness Questionnaires during the assessment. This includes a first indicative comparison of VR and desktop applications for the given use case.