Contrastive learning-based image retrieval for automatic recognition of in situ marine plankton images

Abstract

Abstract Automatic recognition of in situ marine plankton images has long been treated as an image classification problem in machine learning. However, the deep learning-based classifiers are far from robust when used for predicting actual oceanic data that inevitably has distributional and compositional variations from their training sets. This paper proposes a novel image retrieval-based framework for plankton image recognition, within which supervised contrastive learning is used to train a feature extractor for better image representation, and similarity between the input and a gallery of reference images is compared to determine the identity of queries. We construct a dataset of high-quality in situ dark-field images of plankton and suspended particles to train and test the proposed retrieval model. Experimental results show that the image retrieval method has achieved excellent recognition performance similar to the state-of-the-art classification models on a very imbalanced closed-set, and also exhibited better generalizability in dealing with dataset shift and out-of-distribution issues. In addition, the image retrieval method has also demonstrated great architectural flexibility, bringing practical convenience for its adaptation to complex marine application scenarios. This new recognition framework is expected to enable real-time in situ observation of marine plankton in the actual oceanic underwater environment in the near future.