Evaluating language model embeddings for Parkinson's disease cohort harmonization using a novel manually curated...

Area of Science:

• Computational biology
• Medical informatics
• Natural Language Processing

Background:

• Data harmonization is crucial for clinical research but is often time-consuming.
• Language Models (LMs) show promise for text understanding and could streamline this process.

Purpose of the Study:

• To evaluate the effectiveness of LMs in automating data harmonization for clinical use cases.
• To compare LM-based harmonization with traditional fuzzy string matching methods.

Main Methods:

• Developed the PASSIONATE schema for Parkinson's disease (PD) as a ground truth.
• Utilized text embeddings from two LMs for automated cohort harmonization in PD and Alzheimer's disease (AD).
• Compared LM results against a baseline fuzzy string matching approach.

Main Results:

• LM-based text embeddings significantly outperformed fuzzy string matching for data harmonization.
• Achieved over 80% average accuracy for PD cohort harmonization, improving to 96% with extended match neighborhoods.
• Demonstrated high accuracy in harmonizing both AD and PD datasets.

Conclusions:

• Language Models offer a highly accurate method for automated clinical data harmonization.
• Future improvements are expected with the application of domain-specific LMs.
• LM-based harmonization can accelerate clinical research by reducing manual effort.