Evaluating epidemic forecasts in an interval format

Area of Science:

• Epidemiology
• Biostatistics
• Public Health

Background:

• COVID-19 pandemic necessitates accurate forecasting of cases, hospitalizations, and deaths.
• Forecasts are often provided as central predictive intervals, limiting standard evaluation metrics.
• Existing metrics like the logarithmic score require full predictive distributions, which are not always available.

Purpose of the Study:

• To adapt established methods for evaluating quantile and interval forecasts to epidemic predictions.
• To introduce and explain the weighted interval score for assessing probabilistic epidemic forecasts.
• To provide a framework for interpreting forecast accuracy using interval-based metrics.

Main Methods:

• Application of established quantile and interval forecast evaluation techniques.
• Detailed discussion on the computation and interpretation of the weighted interval score.
• Demonstration of weighted interval score's utility in analyzing COVID-19 forecasts.

Main Results:

• The weighted interval score is a proper score that approximates the continuous ranked probability score.
• It generalizes absolute error to probabilistic forecasts.
• The score allows for decomposition into sharpness and penalties for over/underprediction.

Conclusions:

• The weighted interval score is a valuable tool for evaluating epidemic forecasts presented as intervals.
• This method enhances the assessment of forecast reliability in public health.
• It provides a more nuanced understanding of forecast performance beyond simple point estimates.