Robust Automated Harmonization of Heterogeneous Data Through Ensemble Machine Learning: Algorithm Development and...

Area of Science:

• Biomedical Informatics
• Data Science
• Observational Research

Background:

• Large-scale cohort studies provide valuable clinical data but are resource-intensive.
• Multicohort studies offer an alternative by harmonizing data from existing cohorts.
• Variable encoding differences present significant challenges for accurate data harmonization.

Purpose of the Study:

• To introduce SONAR (Semantic and Distribution-Based Harmonization), a novel method for harmonizing variables across cohort studies.
• To facilitate the execution and enhance the utility of multicohort studies.

Main Methods:

• SONAR employs semantic learning from variable descriptions and distribution learning from participant data.
• It generates embedding vectors for variables, using cosine similarity to assess inter-variable relationships.
• The method was developed and validated using data from three National Institutes of Health cohorts, incorporating supervised refinement with gold standard labels.

Main Results:

• The SONAR method demonstrated superior performance in both intracohort and intercohort variable harmonization compared to existing benchmarks.
• Evaluation metrics included area under the curve and top-k accuracy, with SONAR excelling in most comparisons.
• SONAR significantly improved the harmonization of complex concepts that posed difficulties for traditional semantic methods.

Conclusions:

• SONAR effectively achieves accurate variable harmonization within and between cohort studies.
• The method leverages the combined strengths of semantic and distribution-based learning approaches.
• This facilitates more robust and comprehensive multicohort research.