Patients' perception towards large language models in otorhinolaryngology, head and neck surgery: a single-centre...

Large language models (LLMs) are increasingly discussed for use in clinical practice. Beyond their performance, patients' acceptance is crucial for their implementation. We investigated ORL-HNS patients' familiarity with AI/LLMs, use patterns, and trust in LLM-based medical information and recommendations.

In this single-centre prospective survey at a German university hospital, ORL-HNS patients with and without malignant disease completed a 15-item questionnaire.

A total of 123 patients, 20 (16%) with and 103 (84%) without malignant disease, participated in the study. Most patients were familiar with the term AI (96%, n = 118) and LLMs (78%, n = 96). Overall, 72/123 (59%) reported using LLMs. One third (33%, n = 40) retrieved "Health information", rating the LLMs with median Likert scores for comprehensibility 5 [IQR 4, 6], conciseness 5 [IQR 3, 6] and coherence 5 [IQR 3, 6]. However, perceived medical accuracy received a median rating of 4 [IQR 3, 5], significantly lower than comprehensibility (p < 0.05). With respect to the confidence in the recommendations exclusively by LLMs [median 2 (IQR 2, 3.5)] received significantly lower ratings than doctors [median 5 (IQR 5, 6)] and doctors also using LLMs [median 5 (IQR 4, 6)], p < 0.0001 respectively.

ORL-HNS patients are largely familiar with LLMs and frequently use them, but their trust and confidence regarding health information provided by LLMs alone is limited. Patients show the greatest confidence in doctors' recommendations. Yet they reported similar confidence in physician recommendations and physician recommendations supported by LLMs, suggesting that clinician-led LLM use may be acceptable to many patients.