Privacy-Preserving Predictive Modeling: Harmonization of Contextual Embeddings From Different Sources

Area of Science:

• Biomedical Informatics
• Machine Learning
• Data Privacy

Background:

• Data sharing in healthcare is hindered by privacy concerns.
• Contextual embedding models offer data representation without disclosing raw data.
• Combining embeddings from different hospitals is challenging due to differing embedding spaces.

Purpose of the Study:

• To develop a privacy-preserving method for sharing data representations.
• To build a global model from local private data representations.
• To synchronize information across multiple healthcare sources.

Main Methods:

• A novel methodology to harmonize local contextual embeddings into a global model.
• Utilized Word2Vec for generating embeddings and Procrustes for fusing vector models.
• Employed anchor points for aligning different embedding spaces.

Main Results:

• The harmonized global model demonstrated superior predictive accuracy compared to local models.
• Evaluated on predicting the next diagnosis using sequential medical events from MIMIC-III.
• The approach proved effective for both structured and unstructured data.

Conclusions:

• Harmonized local models can serve as a proxy for a global model, aggregating information across institutions.
• Facilitates sharing of unique hospital-specific information, enhancing data fluidity.
• Enables collaborative model building without compromising patient privacy.