Test-Time Data Quality Degradation in Clinical ML: A Systematic Robustness Analysis on MIMIC-IV

Area of Science:

• Clinical informatics
• Machine learning in healthcare
• Data quality assessment

Background:

• Machine learning models are increasingly used for clinical predictions.
• The reliability of these models depends on the quality of input data.
• Data quality degradation can occur in real-world clinical settings.

Purpose of the Study:

• To evaluate the robustness of clinical classification models against data quality degradation.
• To assess the impact of different types of data corruption on model performance.

Main Methods:

• A controlled experimental study was performed using the MIMIC-IV database.
• Five clinical classification models were trained on clean data.
• Dimension-specific data corruptions were applied at test time to simulate data quality degradation.

Main Results:

• Model performance, measured by AUROC and Brier score, declined as data quality degradation increased.
• Completeness, coherence, and validity degradations had a stronger negative impact than precision degradation.
• The reliability of machine learning-based predictions for in-hospital mortality was substantially compromised.

Conclusions:

• Data quality degradation significantly impacts the performance of clinical classification models.
• Ensuring high data quality is crucial for reliable machine learning predictions in healthcare.
• Further research is needed to develop methods for improving model robustness to data quality issues.