Flexible species distribution modelling methods perform well on spatially separated testing data

Abstract

AbstractAimTo assess whether flexible species distribution models that perform well at nearby testing locations still perform strongly when evaluated on spatially separated testing data.LocationAustralian Wet Tropics (AWT), Ontario, Canada (CAN), north‐east New South Wales, Australia (NSW), New Zealand (NZ), five countries of South America (SA), and Switzerland (SWI).Time periodMost species data were collected between 1950 and 2000.Major taxa studiedBirds, mammals, plants and reptiles.MethodsWe compared 10 species distribution modelling methods with varying flexibility in terms of the allowed complexity of their fitted functions [boosted regression trees (BRT), generalized additive model (GAM), multivariate adaptive regression splines (MARS), maximum entropy (MaxEnt), support vector machine (SVM), variants of generalized linear model (GLM) and random forest (RF), and an Ensemble model]. We used established practices for model selection to avoid overfitting, including parameter tuning in learning methods. Models were trained on presence–background data for 171 species and tested on presence–absence data. Training and testing data were separated using both random and spatial partitioning, the latter based on 75‐km blocks. We calculated the average performance and mean rank of the methods (focussing on the area under the receiver operating characteristic and precision‐recall gain curves, and correlation) and assessed the statistical significance of the differences between them.ResultsThe ranking of methods did not change when evaluated on spatially separated testing data. Methods with the strongest predictive performance were nonparametric methods known to be flexible. An ensemble formed by averaging predictions of five pre‐selected modelling methods was the best model in both random and spatial partitioning, followed by MaxEnt and a variant of random forest.Main conclusionsWhilst some modellers expect methods limited to simple smooth functions to predict better spatially separated data, we found no evidence of that using blocks of 75 km. We conclude that flexible models that are tuned well enough to avoid overfitting are effective at predicting to spatially distinct areas.