Using network analysis to identify leverage points based on causal loop diagrams leads to false inference.

Area of Science:

• Systems Science
• Network Theory
• Causal Inference

Background:

• Causal loop diagrams (CLDs) are mental models representing system relationships.
• Network analysis is increasingly used to identify leverage points within CLDs.
• Quantitative identification of leverage points aims to avoid complex computational modeling.

Purpose of the Study:

• To evaluate the justification of using network analysis metrics (betweenness and closeness centrality) for identifying leverage points in CLDs.
• To assess the reliability of these metrics when CLDs represent the same system with differing causal structures.
• To identify conflicts between the assumptions of network analysis metrics and the nature of CLDs.

Main Methods:

• Comparative analysis of network analysis metrics (betweenness and closeness centrality) applied to CLDs.
• Evaluation of metric performance on CLDs with varied inferred causal structures.
• Identification and analysis of conflicts between metric assumptions and CLD characteristics.

Main Results:

• Network analysis metrics provided unreliable results when applied to CLDs representing the same system but with different causal structures.
• Six key conflicts were identified between the assumptions underlying network analysis metrics and the information captured in CLDs.
• The metrics are not equipped to fully account for the nuances of causal relationships depicted in CLDs.

Conclusions:

• The application of betweenness and closeness centrality to identify leverage points from CLDs is premature and potentially incorrect.
• Erroneous identification of leverage points can lead to flawed policy recommendations.
• Alternative quantitative or qualitative approaches better aligned with system dynamics perspectives are needed.