Affiliation:
1. Department of Information Technology, Indian Institute of Information Technology Allahabad, Prayagraj, U.P., India
Abstract
Generating natural language description for visual content is a technique for describing the content available in the image(s). It requires knowledge of both the domains of computer vision and natural language processing. For this, various models with different approaches are suggested. One of them is encoder-decoder-based description generation. Existing papers used only objects for descriptions, but the relationship between them is equally essential, requiring context information. Which required techniques like Long Short-Term Memory (LSTM). This paper proposes an encoder-decoder-based methodology to generate human-like textual descriptions. Dense-LSTM is presented for better description as a decoder with a modified VGG19 encoder to capture information to describe the scene. Standard datasets Flickr8K and Flickr30k are used for testing and training purposes. BLEU (Bilingual Evaluation Understudy) score is used to evaluate the generated text. For the proposed model, a GUI (Graphical User Interface) is developed, which produces the audio description of the output received and provides an interface for searching the related visual content and query-based search.
Subject
Artificial Intelligence,General Engineering,Statistics and Probability
Reference19 articles.
1. Show and tell: Lessons learned from the mscoco image captioning challenge;Vinyals;IEEE Transactions on Pattern Analysis and Machine Intelligence,2016
2. Chu Y.Y. , Yu X. , Sergei L. , Wang M. and Zhengkui , Automatic Image Captioning Based on ResNet50 and LSTM with Soft Attention, Wireless Communications and Mobile Computing 2020 (2020).
3. Multi-Level Policy and Reward-Based Deep Reinforcement Learning Framework for Image Captioning;Xu;in IEEE Transactions on Multimedia,2020
4. Aung S.P. and Win nwe T. , Automatic Myanmar Image Captioning using CNN and LSTM-Based Language Model. 1st Joint SLTU and CCURL Workshop (SLTU-CCURL 2020) At: Marseille, France, 2020 (2020).
5. That ‘internet of things’ thing;Ashton;RFID Journal,2009