MIXED DISCRIMINANT TRAINING OF HYBRID ANN/HMM SYSTEMS FOR ONLINE HANDWRITTEN WORD RECOGNITION

Abstract

Online handwritten word recognition systems usually rely on Hidden Markov Models (HMMs), which are effective under many circumstances, but suffer some major limitations in real world applications. Artificial neural networks (ANN) appear to be a promising alternative, however they failed to model sequence data such as online handwriting due to their variable lengths. As a consequence, by combining HMMs and ANN, we can expect to take advantage of the robustness and flexibility of the HMMs generative models and of the discriminative power of the ANN. Training such a hybrid system is not straightforward, this is why so few attempts are encountered in literature. We compare several different training schemes: maximum likelihood (ML) and maximum mutual information (MMI) criteria in the framework of online handwriting recognition with a global optimization approach defined at the word level. A new generic criterion mixing generative model and discriminant trainings is proposed, it allows to train a multistate TDNN-HMM system directly at the word level. This architecture is based on an analytical approach with an implicit segmentation. To control the implicit segmentation and to initialize correctly the system without bootstrapping with another recognition system, we have defined a process that constraints the segmentation path and a measure called Average Segmentation Rate (ASR). Recognition experiments on the online IRONOFF database demonstrated the interest of the generic training criterion and the control of the implicit segmentation.