The importance of making testable predictions: A cautionary tale

Area of Science:

• Marine ecology
• Fisheries science
• Predictive modeling

Background:

• Accurate prediction of fish populations is crucial for marine resource management.
• Understanding environmental drivers of fish reproduction is key to ecological forecasting.

Purpose of the Study:

• To investigate the correlation between sea surface temperature (SST) events and fish egg abundance.
• To assess the robustness of predictive models in marine ecology through out-of-sample testing.

Main Methods:

• Analysis of 7 years of weekly fish egg abundance data from Scripps Pier, La Jolla, California.
• Correlation analysis (Pearson ρ) between daily spring SST anomalies and peak summer fish egg abundance.
• Out-of-sample prediction for peak summer egg abundance in 2020.

Main Results:

• A high correlation (Pearson ρ > 0.97) was observed between a specific spring SST event and subsequent peak fish egg abundance across multiple species and time intervals.
• An out-of-sample prediction for 2020 failed, despite initial strong correlations.
• Methodological re-examination identified potential over-fitting issues in the predictive model.

Conclusions:

• The study underscores the critical importance of rigorous, testable out-of-sample predictions for validating ecological models.
• A cautionary example is provided regarding the potential for over-fitting and the need for robust model evaluation in predictive ecology.
• Emphasizes the necessity for predictions and models to be publicly accessible for scrutiny and advancement of ecological science.