Related Experiment Video
Updated: Jun 29, 2025
03:53
Author Spotlight: Regenerative Peripheral Nerve Interface (RPNI) Surgery in Postamputation Pain Management
Published on: March 15, 2024
1.6K
ChatGPT Earns American Board Certification in Hand Surgery
Diane Ghanem1, Joseph E Nassar2, Joseph El Bachour2
1Department of Orthopaedic Surgery, The Johns Hopkins Hospital, Baltimore, MD, USA.
Hand Surgery & Rehabilitation
|March 29, 2024
Summary
Artificial Intelligence (AI) language model ChatGPT-4 achieved a 61.98% score on the American Hand Surgery Certifying Examination. While demonstrating capability, AI still requires improvement for complex medical assessments.
Area of Science:
- Medical Education
- Artificial Intelligence in Medicine
- Surgical Examination Assessment
Background:
- Artificial Intelligence (AI) shows promise in healthcare but has inconsistent performance in specialized medical exams.
- Previous evaluations of AI in orthopaedic and hand surgery exams yielded varied results.
Purpose of the Study:
- To evaluate the capability of ChatGPT-4 to pass the American Hand Surgery Certifying Examination.
- Assess AI performance on a high-stakes surgical board examination.
Main Methods:
- ChatGPT-4 was administered the 2019 American Society for Surgery of the Hand (ASSH) Self-Assessment Exam (200 questions).
- Media-based questions were reviewed, and 8 were excluded due to reliance on video or lack of rationalization.
- Descriptive statistics were used to analyze ChatGPT-4's performance, comparing it to physician scores.
Main Results:
- ChatGPT-4 answered 192 questions with an overall accuracy of 61.98%.
- Performance on non-media questions (65.83%) was slightly higher than media-based questions (55.56%).
- ChatGPT-4 excelled in 'vascular' (81.82%) but scored lower in 'bone and joint' (48.54%) and 'neuromuscular' (56.25%) sections.
Conclusions:
- ChatGPT-4 achieved a notable score, indicating significant capability in processing specialized medical examination content.
- AI performance is influenced by examination structure and language, with current limitations in complex clinical judgment.
- AI shows potential as a supportive tool in medical education and decision-making, but cannot replace specialist expertise in nuanced fields like hand surgery.
