Related Experiment Video
Updated: Oct 4, 2025
05:37
An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
2.2K
Testing the predictive accuracy of COVID-19 forecasts
Laura Coroneo1, Fabrizio Iacone2,1, Alessia Paccagnini3,4
1University of York, United Kingdom.
International Journal of Forecasting
|February 7, 2022
Summary
Forecasting COVID-19 deaths is challenging. Ensemble forecasts, combining multiple predictions, show superior accuracy at longer horizons compared to individual teams or simple benchmarks.
Area of Science:
- Epidemiology
- Biostatistics
- Public Health
Background:
- Accurate forecasting of COVID-19 fatalities is crucial for public health preparedness and resource allocation.
- The United States Centers for Disease Control and Prevention (CDC) collects forecasts from multiple teams to inform policy.
Purpose of the Study:
- To evaluate the predictive accuracy of various COVID-19 fatality forecasts collected by the CDC.
- To compare the performance of different forecasting teams against a simple time-series benchmark.
Main Methods:
- Analysis of COVID-19 fatality forecasts submitted to the CDC.
- Comparison of forecast accuracy at different time horizons (1, 3, and 4 weeks ahead).
- Evaluation against a simple time-series benchmark model.
Main Results:
- No individual forecasting team consistently outperformed the benchmark at the short (1-week) horizon.
- Forecasting teams demonstrated improved success and sometimes outperformed the benchmark at longer horizons (3-4 weeks).
- An ensemble forecast, averaging predictions, was among the top-performing models.
Conclusions:
- Collecting diverse forecasts and combining them into an ensemble may offer a more robust approach for health authorities.
- Relying on a limited number of forecasts may be less effective than utilizing a broader, combined prediction strategy.
Related Concept Videos
01:11Residuals and Least-Squares Property
8.0K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
8.0K
01:03Prediction Intervals
2.4K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.4K
01:13Sensitivity, Specificity, and Predicted Value
731
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
731
01:13Accuracy and Errors in Hypothesis Testing
366
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
366
01:18Steps in Outbreak Investigation
245
In the ever-evolving field of public health, statistical analysis serves as a cornerstone for understanding and managing disease outbreaks. By leveraging various statistical tools, health professionals can predict potential outbreaks, analyze ongoing situations, and devise effective responses to mitigate impact. For that to happen, there are a few possible stages of the analysis:
245
01:15Receiver Operating Characteristic Plot
348
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
348