Utility and Limitations of Large Language Models to Simplify Online Content on Generalized Pustular Psoriasis

Area of Science:

• Dermatology
• Medical Informatics
• Artificial Intelligence

Background:

• Online health information (OHI) on generalized pustular psoriasis (GPP) often uses complex language, exceeding a sixth-grade reading level.
• This lack of readability hinders patient comprehension and engagement with crucial health information.

Purpose of the Study:

• To evaluate the effectiveness of three AI large language models (LLMs) - ChatGPT-3.5, ChatGPT-4, and Google Gemini - in simplifying GPP-related OHI.
• To assess whether LLM-generated content maintains the reliability and quality of the original source material.

Main Methods:

• Texts from the top 20 online search results for GPP were identified.
• LLMs reworded these texts to a sixth-grade reading level.
• Readability was assessed using readability indices and the enhanced DISCERN instrument.

Main Results:

• All tested LLMs significantly improved text readability (p<0.01).
• However, LLM-generated content scored lower on the enhanced DISCERN instrument compared to original texts (p<0.01), indicating reduced reliability and quality.
• Statistical analysis confirmed significant differences in readability and DISCERN scores.

Conclusions:

• While AI LLMs can enhance the readability of dermatological OHI, they do not preserve the original content's reliability and quality.
• These findings suggest a need for caution when utilizing LLMs for patient education in dermatology.
• Further research is needed to optimize LLM use for accurate and reliable patient information dissemination.