An Adaptive Homeostatic Algorithm for the Unsupervised Learning of Visual Features

Abstract

The formation of structure in the visual system, that is, of the connections between cellswithin neural populations, is by and large an unsupervised learning process. In the primary visualcortex of mammals, for example, one can observe during development the formation of cells selectiveto localized, oriented features, which results in the development of a representation in area V1 ofimages’ edges. This can be modeled using a sparse Hebbian learning algorithms which alternatea coding step to encode the information with a learning step to find the proper encoder. A majordifficulty of such algorithms is the joint problem of finding a good representation while knowingimmature encoders, and to learn good encoders with a nonoptimal representation. To solve thisproblem, this work introduces a new regulation process between learning and coding which ismotivated by the homeostasis processes observed in biology. Such an optimal homeostasis ruleis implemented by including an adaptation mechanism based on nonlinear functions that balancethe antagonistic processes that occur at the coding and learning time scales. It is compatible witha neuromimetic architecture and allows for a more efficient emergence of localized filters sensitiveto orientation. In addition, this homeostasis rule is simplified by implementing a simple heuristicon the probability of activation of neurons. Compared to the optimal homeostasis rule, numericalsimulations show that this heuristic allows to implement a faster unsupervised learning algorithmwhile retaining much of its effectiveness. These results demonstrate the potential application of sucha strategy in machine learning and this is illustrated by showing the effect of homeostasis in theemergence of edge-like filters for a convolutional neural network.