U-PASS: An uncertainty-guided deep learning pipeline for automated sleep staging

Area of Science:

• Clinical Machine Learning
• Artificial Intelligence in Healthcare
• Uncertainty Quantification

Background:

• Machine learning (ML) is increasingly used in critical healthcare applications.
• Ensuring the safety and reliability of ML systems in healthcare is paramount.
• Uncertainty estimation is key to identifying confident predictions and mitigating risks.

Purpose of the Study:

• To introduce U-PASS, a human-centered ML pipeline for clinical applications.
• To integrate uncertainty estimation throughout the ML lifecycle (data acquisition, training, deployment).
• To enhance ML reliability through expert collaboration and uncertainty communication.

Main Methods:

• U-PASS employs a two-step training process: supervised pre-training and semi-supervised finetuning.
• Uncertainty is estimated at all stages, guiding data optimization and expert feedback.
• The pipeline defers uncertain predictions to clinical experts for review.

Main Results:

• U-PASS was applied to sleep staging in elderly sleep apnea patients.
• Performance improved systematically across all stages of the pipeline.
• Achieved expert-level accuracy of 85%, a 10% increase from the baseline (75%).
• Deferring uncertain samples to experts yielded the most significant performance gain.

Conclusions:

• U-PASS demonstrates a promising approach for integrating uncertainty estimation into clinical ML.
• The pipeline enhances ML reliability and facilitates collaboration with healthcare professionals.
• This method holds potential for unlocking the full capabilities of AI in clinical settings.