Deep-Learning-Based Stereo Matching of Long-Distance Sea Surface Images for Sea Level Monitoring Systems

Abstract

Due to the advantages of coastal areas in the fields of agriculture, transport, and fishing, increasingly more people are moving to these areas. Sea level information is important for these people to survive after extreme sea level events. With the recent improvements in computing and storage capacities, image analysis as a new measuring method is being rapidly developed and widely applied. In this paper, a multi-camera-based sea level height measuring system was built along Japan’s coast and a deep-learning-based stereo matching method has been proposed for this system to complete 3D measurements. In this system, cameras are set with long base distances to ensure the long-distance monitoring system’s precision, which causes a huge difference between the fields of view of the left and right cameras. Since most common network structures complete stereo matching by depth-wise cross-correlation between left and right images, they rely too much on the high-quality rectification of two images and fail on our long-distance sea surface images. We established a feature detection and matching network to realize sea wave extraction and sparse stereo matching for the system. Based on our previous result using the traditional method, the initial disparity was computed to reduce the search range of stereo matching. A training set with 785 pairs of sea surface images and 10,172 pairs of well-matched sea wave images was constructed to supervise the network. The experimental results verified that the proposed method can realize sea wave extraction and mask generation. It can also realize sparse matching of sea surface images regardless of poor rectification.