Clinically informed semi-supervised learning improves disease annotation and equity from electronic health records: a...

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Biomedical Data Science

Background:

• Structured electronic health record (EHR) data, like ICD codes, are often insufficient for detailed disease characterization due to noise and incompleteness.
• This limits the accuracy of datasets for studying disease pathogenesis, progression, and developing robust artificial intelligence (AI) systems.
• Clinical notes offer a rich, underutilized data source for more nuanced patient condition information.

Purpose of the Study:

• To introduce Ci-SSGAN (Clinically Informed Semi-Supervised Generative Adversarial Network), a novel framework designed to reannotate patient conditions using large-scale unlabeled clinical text.
• To improve the accuracy and equity of patient data for research and AI development.
• To demonstrate the framework's utility in a case study involving glaucoma, a condition with known racial and ethnic disparities.

Main Methods:

• Developed Ci-SSGAN, a semi-supervised generative adversarial network framework integrating clinical information.
• Trained the model on a large dataset comprising 349,587 unlabeled ophthalmology notes and 2,954 expert-annotated notes.
• Incorporated demographic conditioning and semi-supervised learning to minimize reliance on expert annotations.

Main Results:

• Ci-SSGAN achieved high performance in the glaucoma case study, with 0.85 accuracy and 0.95 AUROC.
• Demonstrated a significant 10.19% AUROC improvement over traditional ICD-based labels (0.85 AUROC vs. 0.74 AUROC).
• Narrowed performance gaps across demographic subgroups, showing F1 score gains for Black patients (+0.05), women (+0.06), and younger patients (+0.033).

Conclusions:

• Ci-SSGAN effectively leverages unlabeled clinical text for more accurate and equitable patient condition reannotation.
• The framework offers a scalable solution for AI development, reducing the dependency on extensive expert annotations.
• This approach enhances the potential for developing advanced AI systems in healthcare, particularly in resource-constrained settings and for addressing health disparities.