Extraction and processing of intensive care chart data from a patient data management system

Routine clinical data captured in Patient Data Management Systems (PDMS) in intensive care and perioperative settings are an invaluable resource for clinical research. However, the proprietary, fragmented, and transaction-oriented architecture of many systems severely limits secondary data use and requires extensive Extract, Transform, and Load (ETL) processing.

We developed a modular, Python-based ETL framework that enables flexible, domain-specific extraction of high-frequency, multimodal PDMS data. The system provides reusable components for data retrieval, preprocessing, harmonization, and de-identification, allowing extraction methods to be adapted or extended without modifying the core architecture. Each clinical domain is represented through dedicated Pydantic models enforcing consistent output schemas, type constraints, and automated plausibility checks. SQLAlchemy abstracts database access, while structured preprocessing logic resolves common documentation inconsistencies and transforms heterogeneous PDMS entries into standardized representations.

The framework produces reproducible, analysis-ready datasets through a transparent, auditable workflow. An integrated audit logger records extraction parameters, transformations, and derived fields, providing full traceability. Salted, irreversible pseudonymization is embedded directly into the pipeline, supporting compliance with the European General Data Protection Regulation (GDPR; German: Datenschutz-Grundverordnung, DSGVO) and Art. 27 of the Bayerisches Krankenhausgesetz (BayKrG). By encapsulating extraction logic in modular processing units with consistent validation and automated de-identification, the system replaces complex ad hoc queries with standardized, maintainable, and research-ready processes.

The presented framework overcomes substantial technical and regulatory barriers to the secondary use of PDMS data by operationalizing a governance-first extraction pipeline. Its modular architecture encapsulates site-specific PDMS queries in a bounded adapter layer, while keeping validation, pseudonymization, and audit logging portable and reusable across domains and installations. By embedding domain-level validation models, irreversible pseudonymization, and structured auditing, the framework enables reproducible, governance-compliant access to high-frequency intensive care data. Rather than requiring immediate alignment to a common data model, it provides a pragmatic foundation on which semantic and syntactic interoperability can be added incrementally as requirements and resources evolve.