Assessment of Racial Bias within the Risk Analysis Index of Frailty

Our objective was to assess potential racial bias within the Risk Analysis Index (RAI).

Patient risk measures are rarely tested for racial bias. Measures of frailty, like the RAI, need to be evaluated for poor predictive performance among Black patients.

Retrospective cohort study using April 2010-March 2019 Veterans Affairs Surgical Quality Improvement Program and 2010-2019 National Surgical Quality Improvement Program data. The performance of the RAI and several potential variants were compared between Black and White cases using various metrics to predict mortality (180-day for Veterans Affairs Surgical Quality Improvement Program, 30-day for National Surgical Quality Improvement Program).

Using the current, clinical threshold, the RAI performed as good or better among Black cases across various performance metrics versus White. When a higher threshold was used, Black cases had higher true positive rates but lower true negative rates, yielding 2.0% higher balanced accuracy. No RAI variant noticeably eliminated bias, improved parity across both true positives and true negatives, or improved overall model performance.

The RAI tends to predict mortality among Black patients better than it predicts mortality among White patients. As existing bias-reducing techniques were not effective, further research into bias-reducing techniques is needed, especially for clinical risk predictions. We recommend using the RAI for both statistical analysis of surgical cohorts and quality improvement programs, such as the Surgical Pause.