A Robust Residual Shrinkage Balanced Network for Image Recognition from Japanese Historical Documents

Abstract

Kuzushiji, as characters written in a cursive style, had been continuously used for both publishing and handwriting in Japan before the end of the 19th century. However, well into the twentieth century, most modern Japanese lost the ability to read Kuzushiji due to changes in writing systems. Therefore, how to develop advanced machine learning algorithms to identify Japanese historical characters has great significance in preserving cultural knowledge and promoting sustainable development in urban living. Unlike traditional digit image recognition, Kuzushiji image recognition is a more challenging task due to the complex script for cursive handwriting, multiple variations between characters, and imbalanced data. To address such issues, a novel deep learning-based character recognition method called residual shrinkage balanced network (RSBN) is developed for classifying Japanese historical characters. A novel residual shrinkage structure with a soft threshold function is constructed to reduce redundant features using specially designed skip-connected subnetworks. Then, multiple residual shrinkage modules are stacked together to learn hierarchical features from raw input. Furthermore, a novel class rebalancing strategy is further designed to improve learning performance from imbalanced data. The experiments are conducted on Japanese classical literature datasets. The results demonstrate that the proposed method obtains more than 89.01% testing accuracy on Kuzushiji-MNIST dataset, which outperforms the existing deep learning-based character image recognition methods. Moreover, without additional data augmentation techniques, the proposed approach also achieves 97.16% balanced accuracy on Kuzushiji-49 dataset, which is the best one compared with existing published results. When considering the computational performance, the number of trainable parameters of the proposed method is competitive with that of the typical ResNet-18.