Measuring Implicit Bias in ICU Notes Using Word-Embedding Neural Network Models

Area of Science:

• Medical Informatics
• Natural Language Processing
• Health Disparities

Background:

• Human-like biases in nonmedical datasets are known to be transmitted by Natural Language Processing (NLP) algorithms.
• It remains unclear if NLP algorithms applied to medical notes can similarly transmit biases and reinforce health disparities.

Purpose of the Study:

• To identify implicit bias within clinical notes.
• To determine if these biases are stable across different time periods and geographical locations.

Main Methods:

• Utilized unsupervised word-embedding algorithms to quantitatively measure contextual similarity.
• Analyzed ICU notes from University of California, San Francisco (2012-2022) and Beth Israel Deaconess Hospital (2001-2012).
• Assessed the contextual similarity between racial/ethnic descriptors and stigmatizing language (e.g., noncooperative, violence).

Main Results:

• In UCSF notes, Black descriptors showed less contextual similarity to 'violent' words than White descriptors.
• Conversely, in BIDMC notes, Black descriptors exhibited greater contextual similarity to 'violent' words compared to White descriptors.
• UCSF data also indicated Black descriptors were more contextually similar to 'passivity' and 'noncompliance' words than Latinx descriptors.

Conclusions:

• Implicit bias is detectable in Intensive Care Unit (ICU) notes.
• Contextual relationships between racial/ethnic descriptors and stigmatizing language vary significantly based on time and location.
• NLP models trained on clinical data may transmit implicit bias, potentially reinforcing health disparities; active debiasing is crucial for algorithmic fairness in clinical prediction.