Evaluating performance of early warning indices to predict physiological instabilities

Area of Science:

• Biomedical engineering
• Physiological monitoring
• Critical care medicine

Background:

• Patient monitoring algorithms use physiological signals to predict critical events.
• Traditional metrics like sensitivity are insufficient for continuous monitoring systems.
• Assessing warning systems requires evaluating timeliness and false alarms.

Purpose of the Study:

• To present challenges in evaluating new patient monitoring indices.
• To propose a comprehensive framework for assessing warning index performance.
• To include the timeliness of warnings in performance evaluation.

Main Methods:

• Developed a framework considering notification time window and cumulative sensitivity.
• Analyzed warning persistence and distribution of warning times.
• Examined false-alarm rates associated with warnings.
• Applied the framework to an experimental hemorrhage study.

Main Results:

• The proposed framework offers a more complete characterization of warning index performance.
• Timeliness of warnings and warning burden can be differentiated between systems.
• The framework allows for a nuanced comparison of warning systems.

Conclusions:

• Traditional metrics do not fully capture the performance of continuous patient monitoring warning systems.
• The proposed framework provides a more thorough assessment by including timeliness and warning burden.
• This approach enhances the evaluation of predictive algorithms for critical health events.