Me, my doctor and AI: a qualitative interview study on patients' requirements for trustworthy use of AI-CDSS in...

Artificial intelligence-based clinical decision support systems (AI-CDSS) are increasingly entering care and redefining how medical decisions are made. As patients are directly affected by hybrid physician-AI decisions without interacting with the system itself, trust, transparency and involvement in decision-making become important concerns. This study explored patients' requirements for trustworthy use of AI-CDSS in ophthalmology, with particular attention to expectations regarding disclosure of AI use and informational needs.

We conducted semi-structured interviews with 23 ophthalmology patients and caregivers (aged 22-84 years) in Germany between January and March 2026. Participants were recruited via patient organisations, ophthalmology practices, clinics, social media, or purposively, with the aim of obtaining a sample diverse in age, gender and eye conditions. We used inductive qualitative content analysis to examine their perceptions of AI-assisted decision-making, factors shaping trust in AI-supported medical decisions and expectations regarding communication and AI disclosure.

Participants were generally open to the use of AI-CDSS in ophthalmology but had several requirements for trustworthy use of AI-CDSS. These related to three domains: the physician (eg, AI competence, critical evaluation of outputs), the AI system (eg, performance, explanations) and governance of its use (eg, transparency, consent). Participants considered it essential that physicians be able to identify incorrect AI output. Relatedly, the physician emerged as an important trust mediator: some indicated that pre-existing trust in the physician would extend to trust in the use of AI-CDSS, while for others, trust appears to depend on the perceived physician-AI interaction. Transparency about AI use was particularly important in this context. Almost all participants expected to be informed when AI-CDSS were used, although preferences regarding the amount and format of information varied.

The findings suggest that patients' comfort with AI-assisted clinical decision-making depends strongly on the perceived physician-AI interaction and physician-patient communication. Specifically, disclosure appears to be important for informational purposes and for enabling patients to assess the trustworthiness of physician-AI decisions and remain involved in decision-making. As implementation and governance decisions likely shape how patients experience AI-assisted decision-making, they should be made with explicit attention to patient concerns.