Using network analysis to identify leverage points based on causal loop diagrams leads to false inference

Abstract

AbstractNetwork analysis is gaining momentum as an accepted practice to identify which factors in causal loop diagrams (CLDs)—mental models that graphically represent causal relationships between a system’s factors—are most likely to shift system-level behaviour, known as leverage points. This application of network analysis, employed to quantitatively identify leverage points without having to use computational modelling approaches that translate CLDs into sets of mathematical equations, has however not been duly reflected upon. We evaluate whether using commonly applied network analysis metrics to identify leverage points is justified, focusing on betweenness- and closeness centrality. First, we assess whether the metrics identify the same leverage points based on CLDs that represent the same system but differ in inferred causal structure—finding that they provide unreliable results. Second, we consider conflicts between assumptions underlying the metrics and CLDs. We recognise six conflicts suggesting that the metrics are not equipped to take key information captured in CLDs into account. In conclusion, using betweenness- and closeness centrality to identify leverage points based on CLDs is at best premature and at worst incorrect—possibly causing erroneous identification of leverage points. This is problematic as, in current practice, the results can inform policy recommendations. Other quantitative or qualitative approaches that better correspond with the system dynamics perspective must be explored.