Comparative Accuracy of Generative Artificial Intelligence Platforms on Predoctoral Pediatric Dentistry Examination

Area of Science:

• Artificial Intelligence in Education
• Dental Education Technology
• Pediatric Dentistry

Background:

• Generative artificial intelligence (GenAI) is rapidly evolving.
• Assessing the accuracy of GenAI in specialized educational contexts is crucial.
• The performance of GenAI in dental examinations remains largely unexplored.

Purpose of the Study:

• To compare the accuracy of seven GenAI platforms on a predoctoral pediatric dentistry exam.
• To evaluate the influence of question type and AI model training on accuracy.
• To determine the potential of GenAI as an educational tool in pediatric dentistry.

Main Methods:

• Seven GenAI platforms were tested using 100 multiple-choice questions from a pediatric dentistry exam.
• Five platforms were untrained, and two (ChatGPT3.5, ChatGPT4) were trained on evidence-based data.
• Questions were categorized by type (knowledge vs. critical thinking) and subspecialty domain.

Main Results:

• Trained ChatGPT4 achieved the highest accuracy (90%), while untrained Copilot had the lowest (57%).
• Three GenAI models (trained ChatGPT3.5, untrained and trained ChatGPT4) surpassed the 75% passing score.
• GenAI accuracy was not significantly affected by question type or subspecialty domain.

Conclusions:

• Trained and newer GenAI models demonstrate superior accuracy compared to older or untrained models.
• The performance of top-performing GenAI models was comparable to dental students.
• Trained GenAI shows promise as a supplementary resource for predoctoral pediatric dental education.