Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme Classification

Author:

Aggarwal Apeksha1,Sharma Vibhav1,Trivedi Anshul1,Yadav Mayank1,Agrawal Chirag1,Singh Dilbag1,Mishra Vipul1,Gritli Hassène23ORCID

Affiliation:

1. Department of Computer Science and Engineering, Bennett University, Greater Noida 201310, India

2. Higher Institute of Information and Communication Technologies, University of Carthage, Tunis, Tunisia

3. RISC Lab (LR16ES07), National Engineering School of Tunis, University of Tunis El Manar, Tunis, Tunisia

Abstract

Millions of memes are created and shared every day on social media platforms. Memes are a great tool to spread humour. However, some people use it to target an individual or a group generating offensive content in a polite and sarcastic way. Lack of moderation of such memes spreads hatred and can lead to depression like psychological conditions. Many successful studies related to analysis of language such as sentiment analysis and analysis of images such as image classification have been performed. However, most of these studies rely only upon either one of these components. As classifying meme is one problem which cannot be solved by relying upon only any one of these aspects, the present work identifies, addresses, and ensembles both the aspects for analyzing such data. In this research, we propose a solution to the problems in which the classification depends on more than one model. This paper proposes two different approaches to solve the problem of identifying hate memes. The first approach uses sentiment analysis based on image captioning and text written on the meme. The second approach is to combine features from different modalities. These approaches utilize a combination of glove, encoder-decoder, and OCR with Adamax optimizer deep learning algorithms. Facebook Challenge Hateful Meme Dataset is utilized which contains approximately 8500 meme images. Both the approaches are implemented on the live challenge competition by Facebook and predicted quite acceptable results. Both approaches are tested on the validation dataset, and results are found to be promising for both models.

Publisher

Hindawi Limited

Subject

Multidisciplinary,General Computer Science

Cited by 17 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

1. PolyMeme: Fine-Grained Internet Meme Sensing;Sensors;2024-08-23

2. Visual Sentiment Recognition via Popular Deep Models on the Memotion Dataset;2024 IEEE 9th International Conference for Convergence in Technology (I2CT);2024-04-05

3. Comparison of Deep Learning Methods in Detecting Hate Speech in Indonesian Tweets;Proceedings of the 8th International Conference on Sustainable Information Engineering and Technology;2023-10-24

4. Multimodal Religiously Hateful Social Media Memes Classification based on Textual and Image Data;ACM Transactions on Asian and Low-Resource Language Information Processing;2023-09-16

5. Comparative Study on Sentiment Analysis in Image-Based Memes;2023 9th International Conference on Smart Computing and Communications (ICSCC);2023-08-17

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3