Is it Possible to Re-Educate Roberta? Expert-Driven Machine Learning for Punctuation Correction-Reference-Cited by-同舟云学术

Is it Possible to Re-Educate Roberta? Expert-Driven Machine Learning for Punctuation Correction

Published:2023-06-01 Issue:1 Volume:74 Page:357-368
ISSN:1338-4287
Container-title:Journal of Linguistics/Jazykovedný casopis
language:en
Short-container-title:

Author:

Machura Jakub¹,Žižková Hana¹,Frémund Adam²,Švec Jan²

Affiliation:

1. 1 Department of Czech Language, Faculty of Arts , Masaryk University , Brno , Czech Republic

2. 2 Department of Cybernetics, Faculty of Applied Sciences , University of West Bohemia , Pilsen , Czech Republic

Abstract

Abstract Although Czech rule-based tools for automatic punctuation insertion rely on extensive grammar and achieve respectable precision, the pre-trained Transformers outperform rule-based systems in precision and recall (Machura et al. 2022). The Czech pre-trained RoBERTa model achieves excellent results, yet a certain level of phenomena is ignored, and the model partially makes errors. This paper aims to investigate whether it is possible to retrain the RoBERTa language model to increase the number of sentence commas the model correctly detects. We have chosen a very specific and narrow type of sentence comma, namely the sentence comma delimiting vocative phrases, which is clearly defined in the grammar and is very often omitted by writers. The chosen approaches were further tested and evaluated on different types of texts.

Publisher

Walter de Gruyter GmbH

Subject

Linguistics and Language,Language and Linguistics,Linguistics and Language,Language and Linguistics

Link

https://www.sciendo.com/pdf/10.2478/jazcas-2023-0052

Reference14 articles.

1. Benko, V. (2015). Araneum Bohemicum Maius, verze 15.04. Ústav Českého národního korpusu FF UK, Praha 2015. Accessible at: http://www.korpus.cz.

2. Devlin, J. et al. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. ArXiv. /abs/1810.04805 Accessible at: https://doi.org/10.48550/ar-Xiv.1810.04805.

3. Chordia, V. (2021). PunKtuator: A multilingual punctuation restoration system for spoken and written text. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. pages 312–320. Association for Computational Linguistics. Accessible at: https://doi.org/10.18653/v1/2021.eacl-demos.37.

4. Internet Language Reference Book. (2008–2023). Praha: Ústav pro jazyk český AV ČR. Accessible at: https://prirucka.ujc.cas.cz/.

5. Karlík, P. (2017). Vokativ. In M. Nekula et al. (eds.): Nový encyklopedický slovník češtiny. Accessible at: https://www.czechency.org/slovnik/search?action=listpub&search=vokativ.