New Interactive Machine Learning Tool for Marine Image Analysis

Abstract

ABSTRACTDue to advances in imaging technologies, the rate of marine video and image data collection is drastically increasing. Often these datasets are not analysed to their full potential as extracting information for multiple species, such as their presence and surface area, is incredibly time-consuming. This study demonstrates the potential of a new open-source interactive machine learning tool, RootPainter, to analyse large marine image datasets quickly and accurately. The tool was initially developed to measure plant roots, but here was tested on its ability to measure the presence and surface area of the cold-water coral reef associate sponge species,Mycale lingua, in two types of underwater image data: 18,346 time-lapse images and 1,420 remotely operated vehicle video frames. New corrective annotation metrics integrated with RootPainter, such as dice score and species area error, allow for the objective assessment of when to stop model training and reduce the need for manual model validation. Three highly accurateMycale linguamodels were created using RootPainter, as indicated by their average dice score of 0.94 ± 0.06. Model transfer and optimisation aided in the production of two of these models, increasing analysis efficiency from 6 to 16 times faster than manual annotation in Photoshop, for underwater observatory images. Sponge and surface area measurements were extracted from both datasets allowing future investigation of sponge behaviours and distributions. This study demonstrates that interactive machine learning tools and model sharing have the potential to dramatically increase image analysis speeds, collaborative research, and our collective knowledge on spatiotemporal patterns in biodiversity.