Affiliation:
1. Indian Institute of Information Technology Kalyani, India and New Alipore College, India
2. Indian Institute of Information Technology Kalyani, India
3. Sookmyung Women’s University, Rep. of Korea
Abstract
A Part-of-Speech (POS) tagger and Chunker (or shallow parser) are sequence labeling tools, crucial for improving the accuracy of Natural Language Processing (NLP) tasks like parsing, named entity recognition, sentiment analysis, information extraction, and so on. Developing such tools for a low-resource language is an arduous task. Nepali is a relatively resource-poor Indian language and has not been able to evolve from a computational perspective. Therefore, we present effective part-of-speech tagging and chunking tools for the Nepali text using sequential deep learning models—Bidirectional Long Short-Term Memory Network with a Conditional Random Field Layer (BI-LSTM-CRF) and other LSTM-based models exploring both character and word embeddings of the Nepali texts. Word Embedding has been used to capture syntactic as well as semantic information whereas character embedding has been applied to capture the morphological as well as shape information of words and also to handle the out-of-vocabulary problem. The developed chunker is the first statistical chunker for the Nepali language. A baseline model with a Conditional Random Field has also been developed to identify the optimum feature set for the aforementioned tasks. The BI-LSTM-CRF model produced an accuracy of 99.20% and 98.40%, for Nepali POS tagging and chunking, respectively. This is the highest-ever accuracy for Nepali. Thorough error analysis and observations have also been reported with examples. The developed tools can help advance research in Nepali language processing, improve the accuracy of language technology applications, and contribute to the preservation and promotion of the Nepali language.
Publisher
Association for Computing Machinery (ACM)
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. UPoS Tagger for Low Resource Assamese Language: LSTM and BiLSTM based Modelling;2023 IEEE International Conference on Machine Learning and Applied Network Technologies (ICMLANT);2023-12-14