Automatic ETL Pipeline Generation for Mapping Heterogeneous Clinical Data into the OMOP Common Data Model

The OMOP Common Data Model (CDM) standardizes heterogeneous clinical data for large-scale research, yet ETL development remains complex and manually intensive. We present an automated, mapping-driven ETL pipeline that converts CSV-based clinical datasets into OMOP CDM v5.4 using Python-based metadata processing and dbt-generated SQL models. The architecture separates raw, staging, intermediate, and OMOP layers in PostgreSQL and incorporates Rabbit-in-a-Hat-derived mappings and robust handling of CSV formats. By generating all transformation logic programmatically, the pipeline improves reproducibility, transparency, and maintainability. Only mapping definitions require manual creation; all downstream models are synthesized automatically, enabling efficient reuse across datasets, institutions, and domains.