Applying GPT-4 to the Plastic Surgery Inservice Training Examination

Area of Science:

• Artificial Intelligence in Medical Education
• Natural Language Processing Applications
• Surgical Training Technologies

Background:

• Generative Pre-trained Transformer (GPT)-4 is a recent advancement over GPT-3.5, noted for enhanced reliability and creativity.
• The potential of AI language models in graduate medical education requires thorough evaluation.

Purpose of the Study:

• To assess the performance of GPT-4 as an educational tool for plastic surgery residents.
• To evaluate GPT-4's accuracy on the Plastic Surgery Inservice Training Examination (PSITE).

Main Methods:

• Sample questions from the 2022 PSITE were manually inputted into GPT-4.
• GPT-4 responses were analyzed for natural coherence and accuracy.
• Incorrect answers were categorized into informational, logical, or explicit fallacies.

Main Results:

• GPT-4 answered 187 out of 242 questions correctly, achieving a 77.3% accuracy rate.
• GPT-4 demonstrated high utilization of internal (98.3%) and external (97.5%) information, and logical reasoning (95.0%).
• A statistically significant difference was observed in GPT-4's application of logical reasoning between correct and incorrect answers.

Conclusions:

• GPT-4 shows promise as a more accurate and reliable tool for plastic surgery resident education compared to previous versions.
• Integrating GPT-4 into educational curricula may enhance resident preparedness and contribute to high-quality patient care.
• Further exploration of AI tools like GPT-4 is recommended for advancing medical education.