Assessing ChatGPT and Gemini Responses to Common Patient Questions Regarding Augmentation Mammaplasty

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Patient Education

Background:

• Large language models (LLMs) are increasingly used by patients for medical information.
• The quality and readability of LLM-generated medical content require careful evaluation.

Purpose of the Study:

• To compare the quality and readability of ChatGPT and Google Gemini responses to augmentation mammaplasty (AM) frequently asked questions (FAQs).
• To assess the suitability of LLMs for patient education in plastic surgery.

Main Methods:

• Ten AM FAQs were posed to ChatGPT (GPT-4.1 mini) and Google Gemini (2.5 Flash).
• Responses were evaluated for quality by plastic surgeons using the Global Quality Score (GQS).
• Readability was assessed using Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL).

Main Results:

• Both ChatGPT and Gemini produced high-quality answers, with no significant difference in GQS.
• ChatGPT responses were significantly more readable (lower FKGL) than Gemini's.
• Inter-rater agreement was higher for Gemini (Kendall's W = 0.60) compared to ChatGPT (W = 0.24).

Conclusions:

• LLMs like ChatGPT and Gemini can generate high-quality information for augmentation mammaplasty FAQs.
• ChatGPT offers superior readability, crucial for patient comprehension.
• Clinician oversight is essential when using LLMs for patient education to ensure accuracy and prevent misinformation.