Holographic method of neural network design

Abstract

The paper presents a method for solving the problem of changing a full-connected neural network of direct propagation with a sigmoid activation function in the conditions of the emergence of a new class, with the ability to preserve the network to recognize already known classes and to classify objects of a new class. Such conditions generate a new neural network, which is trained on examples of all classes, including the new class. The learning process takes a long time and requires the selection of several parameters. The emergence of a new class in natural neural networks does not cause a transformation of the network structure, only the strength of connections between neurons changes. The network shows the properties of stability and plasticity at the same time. The authors draw attention to the analogy between neural networks and holograms in their ability to store information and form an image of a class in response to an input signal. Following the holographic analogy, the paper proposes a model of the wave nature of neural networks, which treats the network weights as a hologram and the input signal as a wave passing through a hologram. The construction of a new network is created with two neural networks, which are a combination of two holograms. The first hologram represents the original network, and the second is a new network with a similar structure, but it is trained to recognize one new class. The addition of the holograms of these neural networks implements the mechanisms of plasticity and stability in the model.