Federated learning of medical concepts embedding using BEHRT

Area of Science:

• Medical Informatics
• Machine Learning
• Healthcare Data Analytics

Background:

• Electronic health record (EHR) data is sensitive, hindering multicenter data sharing for robust predictive model development.
• Federated learning (FL) enables collaborative model training across multiple locations without centralizing data.

Purpose of the Study:

• To evaluate a federated learning (FL) approach using the BEHRT model for next visit prediction on EHR data.
• To assess the performance of FL compared to centralized and local models for EHR predictive tasks.

Main Methods:

• Proposed an FL approach for learning medical concept embeddings using the BEHRT model.
• Trained both masked language modeling (MLM) and next visit prediction models using FL on the MIMIC-IV dataset.

Main Results:

• The FL approach achieved performance comparable to a centralized model, with average precision differences of 0%-3%.
• FL models demonstrated significant improvement over local models, increasing average precision by 4%-10%.
• Pretrained MLM enhanced the performance of next visit diagnosis prediction tasks.

Conclusions:

• Federated learning (FL) is a viable strategy for developing robust predictive models from distributed EHR data.
• The proposed FL approach, utilizing BEHRT and pretrained MLM, effectively addresses data privacy concerns while achieving high predictive accuracy.