Auditing Learned Associations in Deep Learning Approaches to Extract Race and Ethnicity from Clinical Text

Area of Science:

• Biomedical Informatics
• Natural Language Processing (NLP)
• Health Equity Research

Background:

• Complete and accurate race and ethnicity (RE) patient information is crucial for biomedical informatics.
• Structured RE data is often incomplete, necessitating NLP approaches using clinical notes.
• Existing NLP models may perpetuate bias, associating negative language with specific racial/ethnic groups.

Purpose of the Study:

• To develop and present an approach for auditing NLP models used for RE identification in clinical text.
• To measure the concordance between model-identified features and human-annotated RE spans.
• To identify and address potential biases and harms in RE-identification models.

Main Methods:

• Developed an auditing approach for RE-identification NLP models.
• Measured model-derived salient features against manually identified RE-related text spans.
• Assessed learned associations within NLP models trained on clinical notes.

Main Results:

• NLP models demonstrate surface-level performance in identifying RE information.
• Auditing revealed concerning learned associations within the models.
• Potential for future harms from RE-identification models was identified if biases are not addressed.

Conclusions:

• While NLP models show promise for RE data extraction, their learned associations require careful auditing.
• Unaddressed biases in RE-identification models pose risks for patient care and health equity.
• Further research and development are needed to mitigate bias in clinical NLP applications.