Evaluating Google Gemini's AI generated responses to questions after breast reconstruction

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Surgical Oncology

Background:

• Patient education is crucial for post-operative recovery in breast reconstruction.
• Evaluating the accuracy and accessibility of AI-generated medical information is essential.

Purpose of the Study:

• To assess the quality of Google Gemini's responses to common post-operative breast reconstruction queries.
• To compare expert evaluations of AI-generated patient information.

Main Methods:

• Google Gemini AI was prompted with 14 post-operative breast reconstruction questions.
• Responses were evaluated by surgeons and advanced practice providers (APPs) on accuracy, completeness, relevance, and quality using a 4-point Likert scale.
• Readability was assessed using the Flesch-Kincaid scale.

Main Results:

• The majority of Gemini's responses were rated as accurate (86-79%), complete (86-86%), and relevant (93-100%) by both groups.
• Physicians rated overall quality higher than APPs (p=0.003).
• The mean Flesch-Kincaid readability score was 52.3, indicating "fairly difficult" text.

Conclusions:

• Google Gemini offers relevant and complete information for breast reconstruction post-operative care.
• The AI's readability level may present challenges for some patients.
• While a valuable tool, AI-generated advice requires validation with healthcare professionals for personalized care.