Parsimonious Minimal Learning Machine via Multiresponse Sparse Regression

Abstract

The training procedure of the minimal learning machine (MLM) requires the selection of two sets of patterns from the training dataset. These sets are called input reference points (IRP) and output reference points (ORP), which are used to build a mapping between the input geometric configurations and their corresponding outputs. In the original MLM, the number of input reference points is the hyper-parameter and the patterns are chosen at random. Therefore, the conventional proposal does not consider which patterns will belong to each reference point group, since the model does not implement an appropriate way of selecting the most suitable patterns as reference points. Such an approach can impact on the decision function in terms of smoothness, resulting in high complexity models. This paper introduces a new approach to select IRP for MLM applied to classification tasks. The optimally selected minimal learning machine (OS-MLM) relies on the multiresponse sparse regression (MRSR) ranking method and the leave-one-out (LOO) criterion to sort the patterns in terms of relevance and select an appropriate number of input reference points, respectively. The experimental assessment conducted on UCI datasets reports the proposal was able to produce sparser models and achieve competitive performance when compared to the regular strategy of selecting MLM input RPs.