Evaluating Prediction of Continuous Clinical Values: A Glucose Case Study.

Area of Science:

• Clinical Informatics
• Biomedical Engineering
• Data Science in Healthcare

Background:

• Evaluating predictive models for continuous values is essential prior to clinical trials.
• Accurate forecasting of physiological parameters, such as glucose levels, aids clinical decision-making.

Purpose of the Study:

• To compare various metrics for assessing the clinical utility of continuous value forecasting models.
• To determine the correlation between different performance metrics and their clinical relevance.

Main Methods:

• Applied data assimilation forecast algorithms to glucose time series data from neurological intensive care unit patients.
• Evaluated forecasts using glucose root mean square (RMS) error, transformed glucose metrics, clinical care impact estimation (insulin guideline), and the Parkes grid.
• Assessed metric correlations and developed factor models.

Main Results:

• Most metrics showed correlation, but those estimating clinical care impact correlated least with others.
• Metrics separated into those sensitive to low glucose errors versus high glucose errors.
• The Parkes grid correlated well with transformed glucose metrics but not with clinical care impact estimation.

Conclusions:

• Common metrics like RMS error or the Parkes grid may not accurately reflect the impact on clinical care processes.
• A combination of metrics is likely needed to explain the most variance in model performance.
• Measuring the actual effects of prediction algorithms is vital as they are implemented in practice.