Self-fulfilling prophecies and machine learning in resuscitation science

Area of Science:

• Healthcare AI
• Clinical Informatics
• Machine Learning Ethics

Background:

• The increasing use of machine learning (ML) in healthcare offers potential for data-driven clinical practice.
• However, ML applications can inadvertently create self-fulfilling prophecies (SFPs), where predictions influence outcomes.

Purpose of the Study:

• To explore how machine learning can create, perpetuate, or compound self-fulfilling prophecies (SFPs).
• To identify mechanisms through which SFPs emerge in ML-driven healthcare.
• To offer recommendations for mitigating the risks associated with SFPs in clinical applications.

Main Methods:

• A scoping review of literature from PubMed and ArXiv was conducted.
• Search terms included machine learning, algorithmic fairness, and bias.
• Manuscripts were selected based on expert opinion, focusing on well-designed or key studies and review articles.

Main Results:

• Four key mechanisms for ML-driven SFPs were identified.
• These include encoding imperfect human beliefs, compounding through human-machine interaction, introducing SFPs from incorrect predictions, and perpetuating outdated clinical choices.
• The interplay between ML predictions and clinical actions, especially unrecorded ones, contributes to SFPs.

Conclusions:

• Broad recognition of self-fulfilling prophecies (SFPs) is needed as ML adoption grows in medicine, particularly in resuscitation science.
• Addressing SFPs is vital to transform ML from a tool that risks compounding biases into one that promotes equitable and accurate outcomes.
• Further research and practice are necessary to mitigate the risks of SFPs in ML-driven healthcare.