Study becomes insight: Ecological learning from machine learning

Area of Science:

• Ecological and environmental science
• Data science
• Machine learning applications

Background:

• Machine learning (ML) is widely used for ecological prediction, but extracting functional relationships from models is complex.
• Interpreting ML models requires specialized techniques to uncover underlying ecological insights.
• Existing methods for ML interpretation vary in effectiveness, especially with complex ecological data.

Purpose of the Study:

• To evaluate the effectiveness of different machine learning interpretation methods for ecological inference.
• To investigate the impact of sample size and spurious variables on ML model interpretability.
• To provide guidance on selecting appropriate interpretation techniques for ecological applications.

Main Methods:

• Simulation studies with known functional relationships, noise, and spurious variables.
• Comparison of four variable importance ranking methods: Gini importance (GI), permutation importance (PI), split importance (SI), and conditional permutation importance (CPI).
• Evaluation of two functional relationship inference methods: partial dependence plots (PDP) and accumulated local effect plots (ALE), alongside surrogate models for visualization.

Main Results:

• Interpretation effectiveness is strongly influenced by the chosen algorithm and the presence of spurious variables.
• Removing spurious variables enhances ML model interpretability; increasing sample size offers limited benefit with spurious variables present.
• Split importance (SI) showed slight effectiveness with spurious variables, while Gini importance (GI) and SI were more accurate after removal. Partial dependence plots (PDP) were more effective than accumulated local effect plots (ALE) but sensitive to spurious variables.

Conclusions:

• Machine learning analysts must be cautious about including non-causal correlated variables, as they can hinder ecological inference.
• Prioritizing variables with clear causal links and excluding spurious ones is crucial for reliable ecological insights from ML.
• Effective ecological inference from ML is achievable through judicious method selection, data cleaning, and adequate sample sizes.