Deep learning based instance segmentation of particle streaks and tufts

Abstract

Abstract 3D particle streak velocimetry (3D-PSV) and surface flow visualization using tufts both require the detection of curve segments, particle streaks or tufts, in images. We propose the use of deep learning based instance segmentation neural networks Mask region-based convolutional neural network (R-CNN) and Cascade Mask R-CNN, trained on fully synthetic data, to accurately identify, segment, and classify streaks and tufts. For 3D-PSV, we use the segmented masks and detected streak endpoints to volumetrically reconstruct flows even when the imaged streaks partly overlap or intersect. In addition, we use Mask R-CNN to segment images of tufts and classify the detected tufts according to their range of motion, thus automating the detection of regions of separated flow while at the same time providing accurate segmentation masks. Finally, we show a successful synthetic-to-real transfer by training only on synthetic data and successfully evaluating real data. The synthetic data generation is particularly suitable for the two presented applications, as the experimental images consist of simple geometric curves or a superposition of curves. Therefore, the proposed networks provide a general framework for instance detection, keypoint detection and classification that can be fine-tuned to the specific experimental application and imaging parameters using synthetic data.