Spelling Correction for Text Documents in Bahasa Indonesia Using Finite State Automata and Levinshtein Distance Method

Author:

Christanti Mawardi Viny,Susanto Niko,Santun Naga Dali

Abstract

Any mistake in writing of a document will cause the information to be told falsely. These days, most of the document is written with a computer. For that reason, spelling correction is needed to solve any writing mistakes. This design process discuss about the making of spelling correction for document text in Indonesian language with document's text as its input and a .txt file as its output. For the realization, 5 000 news articles have been used as training data. Methods used includes Finite State Automata (FSA), Levenshtein distance, and N-gram. The results of this designing process are shown by perplexity evaluation, correction hit rate and false positive rate. Perplexity with the smallest value is a unigram with value 1.14. On the other hand, the highest percentage of correction hit rate is bigram and trigram with value 71.20 %, but bigram is superior in processing time average which is 01:21.23 min. The false positive rate of unigram, bigram, and trigram has the same percentage which is 4.15 %. Due to the disadvantages at using FSA method, modification is done and produce bigram's correction hit rate as high as 85.44 %.

Publisher

EDP Sciences

Subject

General Medicine

Reference10 articles.

1. Michelbacher L.. Multi-word tokenization for natural language processing. [PhD dissertation]. University of Stugartt, Stugartt, Germany (2013). pp. 27-28. https://d-nb.info/1046313010/34

2. Global Komputer. Finite automata [Online] from http://www.globalkomputer.com/Bahasan/Teori-Bahasa-dan-Otomata/Topik/Finite-Automata.html (2006). [Accessed on 10 January 2017]. [in Bahasa Indonesia]

3. Smaller representation of finite state automata

4. Zhang W.. Comparing the effect of smoothing and n-gram order: Finding the best way to combine the smoothing and order of n-gram. [PhD dissertation]. Florida Institute of Technology, Melbourne, Florida, pp. 1-6 (2015). https://pdfs.semanticscholar.org/ac6b/aabe681737c3baecf7315b629c7ab4c23577.pdf

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3