A Semantic-Driven for Cohort Data Harmonisation into OMOP CDM Schema.

Area of Science:

• Biomedical Informatics
• Data Science
• Clinical Research Informatics

Background:

• Clinical research data integration faces challenges due to structural, semantic, and linguistic diversity.
• Traditional extract-transform-load (ETL) pipelines lack robust support for semantic variability and multilingual harmonization.
• Data fragmentation hinders the interoperability and reuse of valuable clinical datasets.

Purpose of the Study:

• To propose an integrated framework for harmonizing heterogeneous clinical data.
• To address the limitations of traditional ETL pipelines in handling semantic and linguistic variations.
• To enhance the interoperability and reusability of clinical datasets for large-scale analyses.

Main Methods:

• Developed an embedding-based concept mapping engine utilizing transformer embeddings.
• Integrated the mapping engine with an automated ETL pipeline orchestrated by Apache Airflow.
• Ensured backward interoperability by generating outputs compatible with White Rabbit and Usagi.

Main Results:

• The framework successfully aligned clinical terms with standard concepts using semantic embeddings.
• Demonstrated the system's capability to handle multilingual and heterogeneous real-world clinical datasets.
• Validated end-to-end reproducibility in the data integration process.

Conclusions:

• The proposed integrated framework effectively addresses semantic variability and multilingual challenges in clinical data integration.
• The system enhances data interoperability and facilitates the reuse of clinical datasets.
• This approach offers a reproducible and scalable solution for harmonizing diverse clinical research data.