Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According...

Area of Science:

• Medical Informatics
• Dermatology
• Health Literacy

Background:

• Dermatologic patient education materials (PEMs) often exceed the national average reading level.
• Large language models (LLMs) like ChatGPT-3.5, GPT-4, DermGPT, and DocsGPT show potential for content generation.

Purpose of the Study:

• To assess LLMs' ability to generate dermatologic PEMs at specified reading levels.
• To evaluate the preservation of meaning in LLM-generated PEMs for common and rare dermatologic conditions.

Main Methods:

• Compared Flesch-Kincaid reading levels (FKRL) of existing PEMs with LLM-generated content.
• Prompted four LLMs to create PEMs for common and rare dermatologic conditions at fifth- and seventh-grade FKRLs.
• Assessed meaning preservation through evaluation by dermatology resident trainees.

Main Results:

• Existing PEMs averaged a ninth-grade FKRL.
• LLM-generated PEMs met fifth- and seventh-grade FKRLs, with GPT-4 performing well at fifth grade.
• For seventh-grade level, ChatGPT-3.5 and DocsGPT outperformed GPT-4 for rare conditions; DermGPT showed high accuracy for common conditions.

Conclusions:

• LLMs can generate accessible dermatologic PEMs meeting target reading levels.
• GPT-4, ChatGPT-3.5, DocsGPT, and DermGPT show promise in improving health literacy in dermatology.
• LLM-generated PEMs are largely accurate and understandable, aiding patient comprehension.