Performance of intensive care unit severity scoring systems across different ethnicities

Area of Science:

• Critical Care Medicine
• Health Equity
• Biostatistics

Background:

• Illness severity scoring systems are widely used in intensive care units (ICUs) for case-mix determination and benchmarking.
• Recent guidelines for resource allocation, particularly mechanical ventilation during the COVID-19 pandemic, necessitate examining these scores for potential ethnic bias.
• Previous research has not adequately explored ethnicity-based scoring bias in critical care settings.

Approach:

• The study analyzed data from two large ICU databases: the eICU Collaborative Research Database and the Medical Information Mart for Intensive Care.
• Performance of three common scoring systems (APACHE IVa, OASIS, SOFA) was evaluated across four ethnic groups: Asians, African Americans, Hispanics, and Whites.
• Both discrimination (AUROC) and calibration (SMR) were assessed to identify systematic biases.

Key Points:

• Discrimination, measured by AUROC, showed significant differences among ethnic groups but lacked systematic bias patterns.
• Calibration, assessed by SMR, revealed persistent and significant differences, particularly between Hispanics/African Americans and Asians/Whites.
• Mortality predictions were consistently overestimated for African Americans and Hispanics across the evaluated scoring systems.

Conclusions:

• Systematic differences in calibration across ethnic groups indicate that illness severity scores may reflect inherent biases in their mortality predictions.
• These findings highlight the need for careful consideration of ethnic variations when interpreting and applying ICU scoring systems.
• Further research is warranted to develop and validate bias-corrected scoring tools for equitable critical care resource allocation.