Artificial Intelligence and Human Expertise in Cleft Lip and Palate Care: A Comparative Study of Accuracy, Readability, and Treatment Quality

Nutcha Yodrabum ¹, Sarut Chaisrisawadisuk ^1,2, Sirin Apichonbancha ¹

⁰Division of Plastic Surgery, Department of Surgery, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand.

The Journal of Craniofacial Surgery +

|

June 17, 2025

View abstract on PubMed

Summary

Artificial intelligence (AI) shows promise in cleft care patient education, offering consistent answers on complex topics. However, AI cannot replace experienced surgeons for nuanced clinical judgment and actionable insights.

Area Of Science

Medical Informatics
Artificial Intelligence in Healthcare
Plastic Surgery

Background

Patient education and care standardization are crucial in cleft lip and palate management.
Artificial intelligence (AI) models, like ChatGPT, offer potential tools for information dissemination and communication.
Evaluating AI's performance against human experts is essential for safe and effective integration into clinical practice.

Purpose Of The Study

To compare the accuracy, readability, and quality of AI-generated responses (GPT-3.5, GPT-4, GPT-4o) with human experts for cleft lip and palate inquiries.
To assess AI's utility in patient education and care standardization within craniofacial surgery.
To determine the limitations of AI in providing nuanced clinical judgment and actionable insights.

Main Methods

Comparative analysis of AI and human expert responses to 36 basic and 14 controversial cleft lip and palate questions.
Evaluation using response accuracy, Flesch Reading Ease, Flesch-Kincaid Reading Grade Level, DISCERN, and PEMAT-P scores.
Human expert panel included residents, general plastic surgeons, and craniofacial surgeons.

Main Results

AI demonstrated moderate accuracy (55.6%) on basic questions, below experienced craniofacial surgeons.
AI showed superior consistency and accuracy compared to residents on controversial topics.
GPT-4o excelled in readability and understandability; AI provided higher quality treatment information than less experienced humans (DISCERN scores).
AI struggled with actionable insights and nuanced clinical judgment.

Conclusions

AI, particularly GPT-4o, has potential as a supplementary tool for patient education and standardizing communication in cleft care.
AI cannot replace the clinical expertise and decision-making of experienced craniofacial surgeons.
Integrating AI with human expertise may enhance comprehensive cleft care delivery.

Keywords:

Artificial intelligence cleft lip and palate craniofacial anomalies education