The Exact VC Dimension of the WiSARD n-Tuple Classifier

Abstract

The Wilkie, Stonham, and Aleksander recognition device (WiSARD) [Formula: see text]-tuple classifier is a multiclass weightless neural network capable of learning a given pattern in a single step. Its architecture is determined by the number of classes it should discriminate. A target class is represented by a structure called a discriminator, which is composed of [Formula: see text] RAM nodes, each of them addressed by an [Formula: see text]-tuple. Previous studies were carried out in order to mitigate an important problem of the WiSARD [Formula: see text]-tuple classifier: having its RAM nodes saturated when trained by a large data set. Finding the VC dimension of the WiSARD [Formula: see text]-tuple classifier was one of those studies. Although no exact value was found, tight bounds were discovered. Later, the bleaching technique was proposed as a means to avoid saturation. Recent empirical results with the bleaching extension showed that the WiSARD [Formula: see text]-tuple classifier can achieve high accuracies with low variance in a great range of tasks. Theoretical studies had not been conducted with that extension previously. This work presents the exact VC dimension of the basic two-class WiSARD [Formula: see text]-tuple classifier, which is linearly proportional to the number of RAM nodes belonging to a discriminator, and exponentially to their addressing tuple length, precisely [Formula: see text]. The exact VC dimension of the bleaching extension to the WiSARD [Formula: see text]-tuple classifier, whose value is the same as that of the basic model, is also produced. Such a result confirms that the bleaching technique is indeed an enhancement to the basic WiSARD [Formula: see text]-tuple classifier as it does no harm to the generalization capability of the original paradigm.