The Validity, Accuracy, and Comprehensiveness of Artificial Intelligence for Generating Resident Educational Material...

Area of Science:

• Medical Education
• Surgical Training
• Artificial Intelligence

Background:

• Traditional plastic surgery resident education relies on outdated teacher-learner methods.
• Lectures and passive learning are incorporated into formal conferences.
• The study explored the potential of artificial intelligence (AI) to generate educational content.

Purpose of the Study:

• To evaluate the efficacy of an AI platform in creating plastic surgery resident conference discussion guides.
• To compare AI-generated guides with faculty-developed guides.

Main Methods:

• An AI platform was trained using a bibliography on three surgical topics: breast reduction, frontal sinus fracture, and flexor tendon injury.
• AI-generated discussion guides were assessed by residents on completeness, specificity, and absence of hallucinations.
• Residents also assessed faculty-generated guides using the same criteria.

Main Results:

• AI-generated guides received significantly lower ratings compared to faculty-generated guides.
• AI content was described as lacking substance or being nonsensical.
• Interrater reliability for assessments was 0.85.

Conclusions:

• Current AI technology is not suitable for generating sophisticated educational materials for plastic surgery residents.
• The utility of AI in enhancing resident education requires further investigation and development.