Hybrid Learning of Mapping and its Jacobian in Multilayer Neural Networks

Abstract

There are some learning problems for which a priori information, such as the Jacobian of mapping, is available in addition to input-output examples. This kind of information can be beneficial in neural network learning if it can be embedded into the network. This article is concerned with the method for learning the mapping and available Jacobian simultaneously. The basic idea is to minimize the cost function, which is composed of the mapping error and the Jacobian error. Prior to developing the Jacobian learning rule, we develop an explicit and general method for computing the Jacobian of the neural network using the parameters computed in error backpropagation. Then the Jacobian learning rule is derived. The method is simpler and more general than tangent propagation (Simard, Victorri, Le Cun, & Denker, 1992). Through hybridization of the error backpropagation and the Jacobian learning, the hybrid learning algorithm is presented. The method shows good performance in accelerating the learning speed and improving generalization. And through computer experiments, it is shown that using the Jacobian synthesized from noise-corrupted data can accelerate learning speed.