Moral Engagement and Disengagement in Health Care AI Development

Area of Science:

• Health Informatics
• Artificial Intelligence in Medicine
• Bioethics

Background:

• Machine learning (ML) is increasingly used in healthcare, posing potential risks to patients and systems.
• Regulators propose shifting harm mitigation responsibility to ML developers.
• Understanding developers' perspectives on their ethical obligations is crucial.

Purpose of the Study:

• To explore healthcare ML developers' views on their responsibility for mitigating potential harms.
• To identify facilitators and barriers to ethical ML development.

Main Methods:

• Conducted 40 semi-structured interviews with US-based developers of ML predictive analytics for healthcare.

Main Results:

• Developer perspectives on responsibility varied, with both moral engagement and disengagement observed.
• 70% of participants showed moral engagement, but often only awareness, not full acceptance of responsibility.
• Eight categories of moral disengagement were identified, aimed at minimizing harm or deflecting responsibility.

Conclusions:

• Findings suggest facilitators and barriers to ethical ML development exist.
• Regulatory approaches relying solely on developer responsibility may be insufficient without education and guidance.
• Further support is needed to encourage moral engagement and discourage disengagement among ML developers.