A Method for Improving the Performance of Ensemble Neural Networks by Introducing Randomization into Their Training Data

Abstract

We propose a methodology for training neural networks in which ensembles of under-trained neural networks are used to obtain broadly repeatable predictions, and we augment their performance by disrupting their training, with each neural network in the ensemble being trained on a potentially different data set generated from the base data by a method that we call randomization with full range sampling. Sleep habits in animals are a function of innate and environmental factors that determine the species’ place in the ecosystem and, thus, its requirement for sleep and opportunity to sleep. We apply the proposed methodology to train neural networks to predict hours of sleep from only seven correlated observations in only 39 species (one set of observations per species). The result was an ensemble of neural networks making more accurate predictions (lower mean squared error) and predictions that are more robust against variations in any one input parameter. The methodology presented here can be extended to other problems in which the data available for training are limited, or the neural network is to be applied, post-training, on a problem with substantial variation in the values of inputs (independent variables).