Automatic discard registration in cluttered environments using deep learning and object tracking: class imbalance, occlusion, and a comparison to human review

Abstract

Abstract This paper presents and evaluates a method for detecting and counting demersal fish species in complex, cluttered, and occluded environments that can be installed on the conveyor belts of fishing vessels. Fishes on the conveyor belt were recorded using a colour camera and were detected using a deep neural network. To improve the detection, synthetic data were generated for rare fish species. The fishes were tracked over the consecutive images using a multi-object tracking algorithm, and based on multiple observations, the fish species was determined. The effect of the synthetic data, the amount of occlusion, and the observed dorsal or ventral fish side were investigated and a comparison with human electronic monitoring (EM) review was made. Using the presented method, a weighted counting error of 20% was achieved, compared to a counting error of 7% for human EM review on the same recordings.