Impact of applying super resolution to low resolution face images on the performance of deep neural networks

Abstract

Abstract Face images captured in unconstrained environment differ in various aspects such as expression, illumination, resolution, occlusion, pose etc. which makes face recognition task difficult. The face images captured by the camera from a distance will have low resolution and lack many finer details that makes face recognition a challenging task. Super resolution (SR) is a process of generating high resolution (HR) images from one or more images. In this work, we apply super resolution to low resolution (LR) images of faces to find the impact on the deep models performance. To achieve this, we create dataset with face images captured in unconstrained environment. Later we designed a CNN model with eight layers and trained on the dataset created. Our deep model with low memory requirement and less parameters achieves an accuracy of 99.75% on test dataset and outperforms fine-tuned VGGFace by a small margin. The performance of our deep neural network and fine-tuned VGGFace was observed on low resolution images pre and post-super resolution. The deep neural network-based model available in OpenCV, SRGAN super resolution model and INTER_CUBIC interpolation are used to generate HR images. The HR images generated by OpenCV, SRGAN are better than INTER_CUBIC interpolation. The results show that HR images generated by applying SR to low resolution face images improve the image quality in terms of Mean squared error (MSE), Structural similarity index measure (SSIM) and Peak to signal noise ratio (PSNR). However, the results indicate that improvement in the image quality does not significantly improve performance of deep model.