EA-GAN: Ancient books text restoration model based on example attention

Abstract

Abstract Ancient books are of great significance to historical research and cultural inheritance. Unfortunately, these books have been damaged and corroded in the process of long-term transmission. The restoration and digital preservation of ancient books is a new protection method. Traditional character restoration methods ensure the visual consistency of character images through the character features and the pixels around the damaged hole. However, character structure often causes errors, especially when there is a big hole in critical locations. Inspired by human copywriting behavior, a two-branch structure character restoration network EA-GAN is proposed, which is based on generative adversarial network and fuses reference examples. By referring to the feature of the example character, the damaged character can be repaired accurately even when the damaged area is large. EA-GAN first uses two branches to extract the features of the damaged character and example character respectively. Then, the damaged character is repaired according to their neighborhood information and example character features of different scales in the upsampling stage. To solve the problems that the example features and the fixed character features are not aligned and the convolution receptive field is too small, an Example Attention structure block is proposed to assin ist repair. Qualitative and quantitative analysis experiments are carried out on the self-built dataset MSACCSD and real scene pictures. Compared with the latest repair networks, EA-GAN can get the correct text structure through the guidance of the additional example in the Example Attention block. The peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) value were improved by 9.82% and 1.82%, and the learned perceptual image patch similarity (LPIPS) value calculated by VGG and AlexNet decreased by 35.04% and 16.36% respectively, which obtained better results than the current repair methods. It also has a good repair effect in the face of untrained characters ,which is helpful to the digital preservation of ancient books.