Generating Human-Like Descriptions for the Given Image Using Deep Learning

Abstract

One of the most prominent applications in the field of computer vision and natural language processing research is image captioner. The paper includes an exhaustive review of the literature on image captioning and the implementation using attention-based encoder-decoder model. The process of depicting an image with textual explanations is known as image captioning. The problem has seen extensive use of encoder-decoder frameworks. In this study, Deep Convolutional Neural Network (CNN) for image classification and Recurrent Neural Network (RNN) for sequence modeling are combined to build a single network that creates descriptions of images using the Microsoft Common Objects in Context Dataset (MSCOCO Dataset). Because of RNNs being computationally expensive to train and assess, memory is often restricted to a few items. By highlighting the most important components of an input image, the Attention model had been used to address this issue. The model was developed using Nvidia Quadro RTX5000 GPU (CUDA), which received the Bleu-1 score of 0.5793 for the 100 generated sentences. The captions generated by the model on the testing dataset labeled nearly all of the objects in the image and were sufficiently like the actual captions in the annotations, even on images outside of the testing dataset.