The perils of policy by p-value: Predicting civil conflicts

Abstract

Large-n studies of conflict have produced a large number of statistically significant results but little accurate guidance in terms of anticipating the onset of conflict. The authors argue that too much attention has been paid to finding statistically significant relationships, while too little attention has been paid to finding variables that improve our ability to predict civil wars. The result can be a distorted view of what matters most to the onset of conflict. Although these models may not be intended to be predictive models, prescriptions based on these models are generally based on statistical significance, and the predictive attributes of the underlying models are generally ignored. These predictions should not be ignored, but rather need to be heuristically evaluated because they may shed light on the veracity of the models. In this study, the authors conduct a side-by-side comparison of the statistical significance and predictive power of the different variables used in two of the most influential models of civil war. The results provide a clear demonstration of how potentially misleading the traditional focus on statistical significance can be. Until out-of-sample heuristics — especially including predictions — are part of the normal evaluative tools in conflict research, we are unlikely to make sufficient theoretical progress beyond broad statements that point to GDP per capita and population as the major causal factors accounting for civil war onset.