Large language models improve transferability of electronic health record-based predictions across countries and...

Area of Science:

• Computational biology
• Medical informatics
• Machine learning in healthcare

Background:

• Healthcare systems exhibit variations in medical practices and reporting standards, hindering the transferability of predictive models.
• Embedding medical codes into a shared semantic space can mitigate discrepancies, but real-world applications are limited.

Purpose of the Study:

• To develop and validate a scalable approach, GRASP, that leverages large language model embeddings to enhance the generalizability of prediction models using electronic health record data.
• To improve the prediction of disease onset and all-cause mortality across diverse international healthcare datasets.

Main Methods:

• Developed GRASP, integrating large language model embeddings with a transformer-based prediction model.
• Applied GRASP to predict 21 diseases and all-cause mortality in over one million individuals.
• Trained on UK Biobank data and evaluated in FinnGen (Finland) and Mount Sinai (USA) datasets.

Main Results:

• GRASP demonstrated significant improvements in generalizability, with average ΔC-index improvements of 88% (FinnGen) and 47% (Mount Sinai) compared to language-unaware models.
• GRASP showed higher correlations with polygenic risk scores for 62% of diseases.
• The model maintained robust performance even with unharmonized datasets.

Conclusions:

• Leveraging large language model embeddings offers an effective and scalable solution for enhancing the generalizability of electronic health record-based prediction models.
• GRASP represents a significant advancement in cross-healthcare system model transferability, addressing critical limitations in current predictive modeling.
• The approach shows promise for more reliable and widely applicable clinical prediction tools.