Experimenting Language Identification for Sentiment Analysis of English Punjabi Code Mixed Social Media Text-Reference-Cited by-同舟云学术

Experimenting Language Identification for Sentiment Analysis of English Punjabi Code Mixed Social Media Text

Published:2022-06-10 Issue: Volume: Page:1470-1479
ISSN:
Container-title:Research Anthology on Implementing Sentiment Analysis Across Multiple Disciplines
language:
Short-container-title:

Author:

Bansal Neetika¹,Goyal Vishal²,Rani Simpel³

Affiliation:

1. College of Engineering & Management, India

2. Punjabi University, India

3. Yadavindra College of Engineering, India

Abstract

People do not always use Unicode, rather, they mix multiple languages. The processing of codemixed data becomes challenging due to the linguistic complexities. The noisy text increases the complexities of language identification. The dataset used in this article contains Facebook and Twitter messages collected through Facebook graph API and twitter API. The annotated English Punjabi code mixed dataset has been trained using a pipeline Dictionary Vectorizer, N-gram approach with some features. Furthermore, classifiers used are Logistic Regression, Decision Tree Classifier and Gaussian Naïve Bayes are used to perform language identification at word level. The results show that Logistic Regression performs best with an accuracy of 86.63 with an F-1 measure of 0.88. The success of machine learning approaches depends on the quality of labeled corpora.

Publisher

IGI Global

Reference19 articles.

1. Code Mixing: A Challenge for Language Identification in the Language of Social Media

2. Beesley, K. R. (1988, October). Language identifier: A computer program for automatic natural-language identification of on-line text. In Proceedings of the 29th annual conference of the American Translators Association (p. 54). Academic Press.

3. Microblog language identification: overcoming the limitations of short, unedited and idiomatic text

4. Cavnar, W. B., & Trenkle, J. M. (1994, April). N-gram-based text categorization. In Proceedings of SDAIR-94, 3rd annual symposium on document analysis and information retrieval. Academic Press.

5. DunningT. (1994). Statistical identification of language. Las Cruces, NM: Computing Research Laboratory, New Mexico State University.