Generalized and transferable patient language representation for phenotyping with limited data.

Area of Science:

• Medical Informatics
• Computational Linguistics
• Artificial Intelligence in Healthcare

Background:

• Clinical natural language processing (NLP) benefits from representation learning via transfer learning.
• Developing generalized patient representations from medical language is crucial for clinical NLP tasks.

Purpose of the Study:

• To propose and evaluate a multi-task pre-training and fine-tuning approach for learning patient representations.
• To assess the effectiveness of this approach for low-prevalence phenotypes with limited data.

Main Methods:

• Multi-task pre-training on high-prevalence phenotypes followed by fine-tuning on downstream tasks.
• Application to low-prevalence phenotypes across circulatory, respiratory, and genitourinary diseases.

Main Results:

• Multi-task pre-training enhances learning efficiency and achieves high, robust performance across most phenotypes.
• The method demonstrates consistent effectiveness, performing comparably to or better than other models.

Conclusions:

• The proposed multi-task transfer learning architecture is a robust method for creating generalized and transferable patient language representations.
• This approach significantly improves NLP performance for numerous, including low-prevalence, clinical phenotypes.