Convolutional Neural Network and Optical Flow for the Assessment of Wave and Tide Parameters from Video Analysis (LEUCOTEA): An Innovative Tool for Coastal Monitoring

Abstract

Coastal monitoring is a topic continuously developing, which has been applied using different approaches to assess the meteo-marine features, for example, to contribute to the development of improved management strategies. Among these different approaches, coastal video monitoring coupled with recent machine learning and computer vision techniques has spread widely to assess the meteo-marine features. Video monitoring allows to obtain large spatially and temporally datasets well-distributed along the coasts. The video records can compile a series of continuous frames where tide phases, wave parameters, and storm features are clearly observable. In this work, we present LEUCOTEA, an innovative system composed of a combined approach between Geophysical surveys, Convolutional Neural Network (CNN), and Optical Flow techniques to assess tide and storm parameters by a video record. Tide phases and storm surge were obtained through CNN classification techniques, while Optical Flow techniques were used to assess the wave flow and wave height impacting the coasts. Neural network predictions were compared with tide gauge records. Furthermore, water levels and wave heights were validated through spatial reference points obtained from pre-event topographic surveys in the proximity of surveillance cameras. This approach improved the calibration between network results and field data. Results were evaluated through a Root Mean Square Error analysis and analyses of the correlation coefficient between results and field data. LEUCOTEA system has been developed in the Mediterranean Sea through the use of video records acquired by surveillance cameras located in the proximity of south-eastern Sicily (Italy) and subsequently applied on the Atlantic coasts of Portugal to test the use of action cameras with the CNN and show the difference in terms of wave settings when compared with the Mediterranean coasts. The application of CNN and Optical Flow techniques could represent an improvement in the application of monitoring techniques in coastal environments, permitting to automatically collect a continuous record of data that are usually not densely distributed or available.