Proposal for Using AI to Assess Clinical Data Integrity and Generate Metadata: Algorithm Development and Validation.

Area of Science:

• Medical Informatics
• Data Science
• Health Services Research

Background:

• Evidence-based medicine relies on high-quality clinical data from research and real-world sources.
• Predictive quality algorithms and machine learning are vital for ensuring data integrity and patient safety.
• Reliable clinical data is essential for research reproducibility and deriving insights from practice.

Purpose of the Study:

• To assess the variability in medical data quality within a university hospital's primary clinical systems.
• To provide researchers with insights into data reliability using machine learning-based predictive quality algorithms.
• To develop and validate a template for predicting data quality and integrating this information into metadata.

Main Methods:

• A literature review identified existing automated quality prediction approaches.
• Metadata, including granularity and quality metrics, was collected during data integration into a medical data integration center (MeDIC).
• Machine learning algorithms (Logistic Regression, k-NN, Naive Bayes, Decision Tree, Random Forest, XGBoost, SVM) were trained and evaluated on echocardiographic, laboratory, and medication data.

Main Results:

• Extreme Gradient Boosting (XGB) achieved an 84.6% AUC-ROC for echocardiographic data quality prediction.
• Support Vector Machines (SVM) demonstrated superior performance for laboratory data with an 89.8% AUC-ROC.
• SVM also provided the most balanced performance for medication data, yielding a 65.1% AUC-ROC.

Conclusions:

• A novel template for predicting data quality and integrating it into metadata within a data integration center was proposed.
• The developed model, combined with conventional methods, was deployed for effective data inspection.
• This approach enhances the trustworthiness and utility of clinical data for research and clinical decision-making.