Clinically relevant pretraining is all you need.

Area of Science:

• Biomedical Informatics
• Artificial Intelligence in Healthcare
• Computational Linguistics

Background:

• Clinical notes contain valuable data but exhibit heterogeneity across institutions and settings.
• Natural Language Processing (NLP) and deep learning models offer solutions for processing clinical text.
• Generalizability of pretrained models across clinical variations is not fully understood.

Purpose of the Study:

• To investigate whether institution or setting-specific pretraining is required for deep learning models in clinical NLP.
• To determine if pretrained models generalize effectively across diverse clinical environments.

Main Methods:

• Evaluated the performance of pretrained models transferred to new clinical NLP tasks.
• Compared models pretrained across various institutions and settings versus those pretrained specifically within single sites.
• Assessed statistical significance of performance differences.

Main Results:

• No significant performance differences were observed between models pretrained across diverse institutions/settings and those pretrained narrowly.
• Clinically pretrained models demonstrate robust transferability across institutional and setting boundaries.
• The necessity of time-consuming, specialized pretraining for new clinical tasks was questioned.

Conclusions:

• Institution or setting-specific pretraining is not essential for achieving high performance in transferred clinical NLP tasks.
• Researchers can leverage existing clinically pretrained models, potentially bypassing extensive re-pretraining.
• Findings support the broad applicability of general clinical models, streamlining NLP research and application development.