Inverted pyramid frame forward and backward prediction for distorted video by water waves

Abstract

There has been much research on how to restore a single image from distorted video. Random water surface variation, an inability to model the surface, and multiple factors in the imaging processing leading to different geometric distortions in each frame are among the challenges. This paper proposes an inverted pyramid structure based on the cross optical flow registration approach and a multi-scale weight fusion method based on wavelet decomposition. The inverted pyramid based on the registration method is used to estimate the original pixel positions. A multi-scale image fusion method is applied to fuse the two inputs processed by optical flow and backward mapping, and two iterations are proposed to improve the accuracy and stability of the output video. The method is tested on several reference distorted videos and our videos, which were obtained through our experimental equipment. The obtained results exhibit significant improvements over other reference methods. The corrected videos obtained with our approach have a higher degree of sharpness, and the time required to restore the videos is significantly reduced.