Electrocardiogram Abnormality Classification From Summary Statistics With Patient-Level Validation and...

Area of Science:

• Clinical Medicine
• Artificial Intelligence
• Biomedical Engineering

Background:

• Machine learning (ML) in clinical medicine faces data leakage risks, including temporal and patient-level issues.
• These vulnerabilities compromise the reliability of electrocardiogram (ECG) abnormality detection systems.
• This study addresses data leakage by employing patient-level data splitting and systematic evaluation.

Purpose of the Study:

• To evaluate the impact of data leakage on ML-based ECG abnormality detection.
• To assess model generalization and stability across different classification scenarios using patient-level data splitting.
• To propose improved evaluation standards for reliable clinical deployment of AI in ECG analysis.

Main Methods:

• Analyzed 4419 ECG observations from 2180 patients using Random Under-Sampling Boosting (RUSBoost).
• Defined three binary classification scenarios for ECG abnormality detection.
• Implemented patient-level stratified partitioning (60:20:20) to prevent data leakage and evaluated models using accuracy, sensitivity, specificity, F1-score, learning curves, and precision-recall curves.

Main Results:

• Scenario 1 (Abnormal vs. Normal) showed optimal generalization (70.79% test accuracy) with monotonic decline across partitions.
• Scenario 2 (Abnormal + Borderline vs. Normal) achieved 65.26% test accuracy with greater performance degradation.
• Scenario 3 (Abnormal vs. Normal + Borderline) exhibited non-monotonic patterns and premature convergence, indicating generalization challenges.

Conclusions:

• Aggregate performance metrics are insufficient for clinical deployment decisions in AI-powered medical tools.
• Thorough evaluation of medical AI must include learning dynamics, generalization stability, and precision-recall calibration across data partitions.
• Process-focused standards, assessing monotonic decline and convergence, are crucial for reliable ECG abnormality detection systems.