Longitudinal Masked Representation Learning for Pulmonary Nodule Diagnosis from Language Embedded EHRs

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Data Science

Background:

• Electronic health records (EHRs) contain valuable longitudinal clinical data but are challenging to utilize for pulmonary nodule diagnosis due to noise and abstraction.
• Scarcity of labeled data and model overfitting are significant hurdles in developing accurate diagnostic classifiers for pulmonary nodules using EHRs.

Purpose of the Study:

• To investigate masked representation learning (MRL) as a method to improve pulmonary nodule diagnosis by modeling longitudinal EHR data.
• To assess the efficacy of MRL in integrating multiple EHR modalities including clinical conditions, procedures, and medications.

Main Methods:

• Leveraged a web-scale text embedding model to encode EHR event streams into semantically embedded sequences.
• Pretrained a bidirectional transformer using MRL conditioned on time encodings on a large cohort of general pulmonary conditions.
• Evaluated the MRL-finetuned model on a cohort of diagnosed pulmonary nodules, comparing its accuracy to a standard supervised model.

Main Results:

• The MRL-finetuned model achieved a higher diagnosis accuracy (0.781 AUC) compared to the supervised model (0.768 AUC) when integrating clinical conditions, procedures, and medications.
• Demonstrated significant improvement in pulmonary nodule diagnosis accuracy through the application of MRL on longitudinal EHR data.
• The study highlights the effectiveness of language-embedded MRL in enhancing downstream clinical classification tasks.

Conclusions:

• Masked representation learning (MRL) is a promising strategy for improving pulmonary nodule diagnosis from longitudinal EHR data.
• Integrating multiple EHR modalities through language-embedded MRL enhances model performance and offers potential advancements in clinical data analysis.
• This approach can help overcome challenges associated with data scarcity and model overfitting in clinical classification.