Recovering Microscopic Images in Material Science Documents by Image Inpainting

Abstract

Microscopic images in material science documents have increased in number due to the growth and common use of electron microscopy instruments. Through the use of data mining techniques, they are easily accessible and can be obtained from documents published online. As data-driven approaches are becoming increasingly common in the material science field, massively acquired experimental images through microscopy play important roles in terms of developing an artificial intelligence (AI) model for the purposes of automatically diagnosing crucial material structures. However, irrelevant objects (e.g., letters, scale bars, and arrows) that are often present inside original microscopic photos should be removed for the purposes of improving the AI models. To avoid the issue above, we applied four image inpainting algorithms (i.e., shift-net, global and local, contextual attention, and gated convolution) to a learning approach, with the aim of recovering microscopic images in journal papers. We estimated the structural similarity index measure (SSIM) and ℓ1/ℓ2 errors, which are often used as measures of image quality. Lastly, we observed that gated convolution possessed the best performance for inpainting the microscopic images.