Evaluating the quality of artificial intelligence responses to psoriasis-related clinical and patient questions: a...

Patient use of artificial intelligence (AI) chatbots for dermatologic information is increasing, but their performance on psoriasis-related questions across clinically distinct domains remains unclear. We compared ChatGPT (GPT-5.3 Instant), Gemini (Gemini 3 Flash), and Microsoft Copilot using a multidimensional scoring framework.

Fifty-four psoriasis-related questions were submitted to each model across diagnostic (n = 12), treatment (n = 12), and patient-question (n = 30) categories. Three board-certified dermatologists independently scored responses for accuracy, evidence consistency, completeness, and clinical safety (maximum score, 8).

Interrater agreement was substantial to almost perfect (κ = 0.743 for ChatGPT, 0.830 for Gemini, and 0.844 for Copilot). Overall mean scores differed significantly: 7.36 ± 0.97, 7.77 ± 0.77, and 7.05 ± 1.09, respectively (Friedman P < 0.001). No difference was observed for diagnostic questions (P = 0.37). Gemini outperformed both models in treatment (8.00 ± 0.00) and patient questions (P < 0.001), while ChatGPT and Copilot did not differ in treatment. Differences were driven by completeness and accuracy, not clinical safety or evidence consistency. Gemini also had the highest rate of high-reliability responses (92.6%).

All models showed high clinical safety, but Gemini provided the most complete and highest-quality responses. The observation that models may provide accurate yet clinically incomplete responses, particularly for treatment content, emphasizes the need for physician oversight when AI-generated information is used in dermatological practice.