Improving the localisation of features for the calibration of cameras using EfficientNets

Abstract

Camera-based methods for optical coordinate metrology, such as digital fringe projection, rely on accurate calibration of the cameras in the system. Camera calibration is the process of determining the intrinsic and distortion parameters which define the camera model and relies on the localisation of targets (in this case, circular dots) within a set of calibration images. Localising these features with sub-pixel accuracy is key to providing high quality calibration results which in turn allows for high quality measurement results. A popular solution to the localisation of calibration features is provided in the OpenCV library. In this paper, we adopt a hybrid machine learning approach where an initial localisation is given by OpenCV which is then refined through a convolutional neural network based on the EfficientNet architecture. Our proposed localisation method is then compared with the OpenCV locations without refinement, and to an alternative refinement method based on traditional image processing. We show that under ideal imaging conditions, both refinement methods provide a reduction in the mean residual reprojection error of approximately 50%. However, in adverse imaging conditions, with high noise levels and specular reflection, we show that the traditional refinement degrades the results given by pure OpenCV, increasing the mean residual magnitude by 34%, which corresponds to 0.2 pixels. In contrast, the EfficientNet refinement is shown to be robust to the unideal conditions and is still able to reduce the mean residual magnitude by 50% compared to OpenCV. The EfficientNet feature localisation refinement, therefore, enables a greater range of viable imaging positions across the measurement volume. leading to more robust camera parameter estimations.