Ethically aligning a cardiovascular AI-DSS to exclude coronary artery disease with the clinical environment: a...

Perspectives and experiences of potential end users are essential to facilitate the ethical implementation of medical Artificial Intelligence (AI) models. In this study, we explored clinician and patient perspectives on the use and ethically relevant aspects of an AI-Decision Support System (AI-DSS), the ARGUS algorithm, developed for the exclusion of Coronary Artery Disease (CAD).

We performed a qualitative study with semi-structured interviews to explore the perspectives of end users regarding the ARGUS AI-DSS. Cardiologists (n=11) and cardiology patients (n=20) were interviewed to study their expectations of this AI-DSS, the conditions under which they trust the outcome of the system and under which conditions they are willing to use it. All interview recordings were transcribed verbatim and a thematic analysis was conducted on the transcribed data.

Respondents in our study generally recognised the added value of using an AI-DSS targeted at excluding CAD as a supplement to clinical judgement. When discussing the anticipated benefits, both cardiologists and patients seemed to prioritise increased accuracy and efficiency while easing the patient burden was mentioned less frequently. One key concern raised by many cardiologists and patients-particularly aimed at AI systems focused on disease exclusion-is whether a low likelihood of CAD provides sufficient reassurance to patients; many may still choose to undergo a scan regardless of the algorithm's recommendation. Further, respondents mentioned that the AI-DSS can lead to responsibility and trust shifts in clinical practice that are essential to consider before and during use.

As disease exclusion becomes an emerging focus in AI development, the findings of this study offer key insights into how these can be successfully and ethically integrated into clinical practice. Additionally, the results provide important empirical insights that are relevant for ongoing ethical discussions regarding medical AI, namely: (1) the need for sex-specific AI-DSSs in cardiology, (2) shared responsibility for AI in clinical practice and (3) overruling the AI system in cases of human-AI disagreement.