Application of the permutation method to the assessment of predictive ability of the models of spatial distribution of copper and iron concentrations in the topsoil

Abstract

The article proposes the use of the permutation method for assessment of the predictive ability of models based on artificial neural networks. To test this method, three models based on artificial neural networks were implemented: a multilayer perceptron, a radial basis function network, and a generalized regression neural network. For modeling, data on the spatial distribution of copper and iron in the topsoil (depth 0.05 m) on the territory of the subarctic city of Noyabrsk, Yamalo-Nenets Autonomous Okrug, Russia, were used. A total of 237 soil samples were collected. For modelling, the copper and iron concentration data were divided into two subsets: training and test. The modelled spatial datasets were compared with the observed values of the test subset. To assess the performance of the constructed models, three approaches were used: 1) calculation of correlation coefficients, error or agreement indexes, 2) graphical approach (Taylor diagram), 3) randomization assessment of the probability of obtaining a divergence between the observed and modelled datasets, assuming that both of these datasets taken from the same population. For the randomization algorithm, two statistics were used: difference in means and correlation coefficient. The permutation method proved its productivity, as it allowed to assess the significance of the divergence between the observed and predicted datasets.