Influence of model complexity, training collinearity, collinearity shift, predictor novelty and their interactions on ecological forecasting

Abstract

AbstractAimEcological forecasting is critical in understanding of ecological responses to climate change and is increasingly used in climate mitigation plans. The forecasts from correlative models can be challenged by model complexity, training collinearity, collinearity shift and novel conditions of predictors that are common during model extrapolation. The individual effect of these four factors has been investigated, but it is still unclear how these four factors interactively affect forecasting. To fill this gap, we conducted a comprehensive simulation experiment to quantify how the four factors interactively influence model forecasting.LocationSimulated regions.Time PeriodSimulated scenarios.MethodsWe modelled three response variables commonly used in ecological forecasting following normal, Poisson and binomial distributions as a function of three functional relationships that represented model complexity under three levels of training collinearity using generalized linear models. By calculating prediction error under 3,780,000 testing scenarios, we partitioned its variance to model complexity, training collinearity, collinearity shift, predictor novelty and their interactions.ResultsWe found that increased predictor novelty and collinearity shift degraded model performance, leading up to double prediction errors when a predictor's range increased by ~22% or when the correlation r between two predictors changed >~0.8 for the combination of high training collinearity and interaction functional relationship. Predictor novelty reduced the influence of collinearity shift on model performance, suggesting a negative interaction between them. This pattern was more pronounced under high model complexity and high training collinearity.Main ConclusionsThe accuracy of ecological forecasting using correlative models depends on model complexity, training collinearity, collinearity shift, predictor novelty and their interactions. Besides the consideration of parsimonious models and r of 0.7 in model training, our study further recommends a threshold of <22%–50% increased predictor range depending on training collinearity and/or <0.8 correlation change for making reliable forecasting.