Early warning signals and the prosecutor's fallacy.

Area of Science:

• Ecology
• Complex Systems Science
• Statistical Modeling

Background:

• Critical transitions, like lake eutrophication or coral reef collapse, are difficult to study experimentally due to scale.
• Historical observations are often used as natural experiments to understand these shifts.
• Existing methods for forecasting transitions may be prone to statistical biases.

Purpose of the Study:

• To investigate the statistical bias, known as the prosecutor's fallacy, in early warning signals for critical transitions.
• To compare the reliability of common warning signal statistics with a model-based approach.
• To identify methods that can accurately forecast impending critical transitions.

Main Methods:

• Analysis of simulated systems exhibiting transitions by chance.
• Evaluation of common summary statistics used as early warning signals.
• Development and testing of a model-based forecasting approach.
• Comparison of false-positive rates between different statistical methods.

Main Results:

• Commonly used early warning signal statistics show an elevated rate of false-positives when systems are selected based on observed transitions.
• The prosecutor's fallacy inflates the perceived reliability of these signals.
• A model-based approach demonstrates reduced susceptibility to this statistical bias.

Conclusions:

• Careful consideration of selection bias is crucial when interpreting early warning signals for critical transitions.
• Model-based approaches offer a more robust framework for forecasting ecological and climate shifts.
• Experimental studies with replicates provide the most reliable data, avoiding these statistical pitfalls.