Extraction and processing of intensive care chart data from a patient data management system

Area of Science:

• Biomedical Informatics
• Data Science
• Clinical Research Technology

Background:

• Patient Data Management Systems (PDMS) in intensive care and perioperative settings contain valuable clinical research data.
• Proprietary and fragmented PDMS architectures hinder data accessibility and necessitate extensive Extract, Transform, and Load (ETL) processing for secondary use.

Purpose of the Study:

• To develop a modular, Python-based ETL framework to overcome barriers in secondary use of PDMS data.
• To enable flexible, domain-specific extraction and standardization of high-frequency, multimodal PDMS data for clinical research.

Main Methods:

• Developed a modular Python ETL framework with reusable components for data retrieval, preprocessing, harmonization, and de-identification.
• Utilized Pydantic models for domain-specific data representation, enforcing schemas, type constraints, and plausibility checks.
• Employed SQLAlchemy for database abstraction and structured preprocessing logic to standardize heterogeneous PDMS entries.

Main Results:

• The framework produces reproducible, analysis-ready datasets via a transparent, auditable workflow with integrated logging for traceability.
• Implemented salted, irreversible pseudonymization for GDPR and BayKrG compliance.
• Replaced complex ad hoc queries with standardized, maintainable, and research-ready processes through modular extraction units.

Conclusions:

• The governance-first ETL pipeline overcomes technical and regulatory barriers to secondary PDMS data use.
• Modular architecture allows for reusable validation, pseudonymization, and audit logging across domains and installations.
• Provides a pragmatic foundation for reproducible, governance-compliant access to high-frequency intensive care data, enabling incremental interoperability.