A qualitative study with patients, care-partners, clinicians, and bioethicists to identify ethical considerations of...

The use of artificial intelligence (AI) in medicine has surged. Given the sensitive nature of palliative care, it is crucial to apply AI tools in a patient-centered and ethical manner.

We sought to understand the ethical implications of implementing AI-based tools in palliative care from various stakeholders.

Using a framework of ethics of care, we conducted thematic analysis to identify emerging themes.

We recruited and interviewed 22 participants: six patients, four care-partners, five clinicians, three bioethicists, and four clinician-bioethicists, recruited through a single hospital clinic for patients and care-partners and purposeful snowball sampling for clinicians and bioethicists based on their knowledge and interest in AI-based tools. The mean age of participants was 48 years old; 64% were male. Clinician participants had practiced for a mean of 11 years, and ethicists for 15 years. We aimed to recruit clinicians and bioethicists from multiple states and specialties.

From the interviews we extracted five main themes: (1) Primacy of the doctor-patient relationship over AI performance; (2) Humans have intuition and nuance that AI lacks; (3) Agreement about the importance of oversight of AI tools; (4) New AI technologies should include a process for patient education; and (5) AI increases efficiency, scalability, and a more unified approach to serious illness.

When building and implementing AI-based tools, we recommend: establishing oversight committees; reflecting on the unique contributions of humans to care; proactively educating patients and contextualizing the tools; and ensuring data use is restricted to clinical care.