Building English – Punjabi Aligned Parallel Corpora of Nouns from Comparable Corpora-Reference-Cited by-同舟云学术

Building English – Punjabi Aligned Parallel Corpora of Nouns from Comparable Corpora

Published:2023-12-01 Issue:2 Volume:28 Page:245-251
ISSN:2255-8691
Container-title:Applied Computer Systems
language:en
Short-container-title:

Author:

Kaur Dilshad¹^ORCID,Singh Satwinder¹^ORCID

Affiliation:

1. 1 Department of Computer Science and Technology , Central University of Punjab Bathinda , Punjab , India

Abstract

Abstract Comparable corpora are the right resources for extracting parallel data due to their abundant availability. It is of great importance where parallel data are scarce. In this study, the focus is placed on building of parallel data for Punjabi and English language pair. The raw data were collected from web contents of “Mann Ki Baat”, which is a collection of textual speeches of Prime Minister of India Mr. Narendra Modi broadcasted every last Sunday of the month. Data were cleaned and pre-processed using a natural language toolkit. An alignment model using BERT was built that aligned two textual files on a sentence level. Furthermore, extraction of noun forms with the help of NLTK library in Python programming was performed. The noun aligned dataset was built for English-Punjabi language pair and made available at Mendeley data repository.

Publisher

Walter de Gruyter GmbH

Subject

General Materials Science

Link

https://www.sciendo.com/pdf/10.2478/acss-2023-0024

Reference33 articles.

1. A. Ali, S. Siddiq, and M. K. Malik, “Development of parallel corpus and English to Urdu statistical machine translation,” Resource, vol. 9, no. 10, 2010. [Online}. Available: https://www.academia.edu/31197083/Development_of_Parallel_Corpus_and_English_to_Urdu_Statistical_Machine_Translation

2. R. Srivastava and R. A. Bhat, “Transliteration systems across Indian languages using parallel corpora,” in Proceedings of the 27th Pacific Asia Conference on Language, Information and Computation (PACLIC 27), 2013, pp. 390–398.

3. M. M. Kenning, “What are parallel and comparable corpora and how can we use them,” in The Routledge handbook of corpus linguistics. Routledge, Jan. 2010, pp. 487–500. https://www.researchgate.net/publication/265061773_What_are_parallel_and_comparable_corpora_and_how_can_we_use_them

4. D. Kaur and S. Singh, “A systematic literature review on extraction of parallel corpora from comparable corpora,” Journal of Computer Science, vol. 17, no. 10, pp. 924–952, Oct. 2021. https://doi.org/10.3844/jcssp.2021.924.952

5. D. Ştefănescu and R. Ion, “Parallel-Wiki: A collection of parallel sentences extracted from Wikipedia,” in Proceedings of the 14th International Conference on Intelligent Text Processing and Computational Linguistics (CICLING 2013), Mar. 2013, pp. 24–30.