Interpretable early warnings using machine learning in an online game-experiment

Area of Science:

• Complex Systems Science
• Computational Social Science
• Statistical Physics

Background:

• The theory of critical transitions posits that regime shifts are often preceded by statistical early warning signals.
• Reddit's r/place experiment offers a unique, large-scale platform to study these signals across numerous subsystems.

Purpose of the Study:

• To develop and validate a machine learning-based early warning system for detecting critical transitions.
• To assess the generalizability and interpretability of such a system in a dynamic social context.

Main Methods:

• A machine learning model combining multiple time series via gradient-boosted decision trees with memory-retaining features was developed.
• The system was trained on 2022 r/place data and tested on 2023 data.
• SHapley Additive exPlanations (SHAP) were used for model interpretation.

Main Results:

• The developed system detected 50% of transitions within 20 minutes with a 3.6% false positive rate.
• Performance remained robust across different years (2022 and 2023), demonstrating generalizability.
• Interpretable drivers included critical slowing/speeding, lack of innovation/coordination, turbulent histories, and reduced image complexity.

Conclusions:

• Machine learning indicators show significant potential for predicting regime shifts in complex systems, particularly online social systems.
• Understanding precursor patterns can enhance the prediction and management of critical transitions in socio-ecological contexts.