Improving the Performance of Frequently Used Korean Handwritten Character Verification Based on Artificial Intelligence through Multimodal Fusion

Abstract

Handwriting verification is a biometric recognition field that identifies individuals’ unique characteristics contained in their handwriting. A single written character shows subtle differences depending on habits accumulated over time or the manner of writing. Based on this, it is often adopted in forensic investigations and as evidence in court. Existing handwriting verification is conducted by an expert, and is affected by the expert’s ability or subjectivity, causing different results to arise depending on the expert. Therefore, we propose a handwriting verification method that excludes human subjectivity and has objectivity. Using computer vision and artificial intelligence (AI), we derived results that excluded human subjectivity, and the judgment strength was expressed through a likelihood ratio. To improve the existing method’s accuracy, we performed a more accurate verification through multimodal use from the biometric field. Multimodal handwriting verification is conducted using up to four characters (not just one) because individual handwriting in each character is different. For learning, n-fold tests were conducted to maintain test objectivity, and the average performance of single character-based verification was 80.14% and the multimodal method averaged 88.96%. Here, we proposed the objectivity of handwriting verification through learning using AI, and show that performance improved through multimodal fusion.