Accuracy of Large Language Models in Frequently Answering Questions Related to Presurgical Nasoalveolar Molding

ObjectiveTo assess the accuracy, readability, and comparative quality of five large language models (LLMs) in answering frequently asked questions related to nasoalveolar molding (NAM) in cleft care.DesignRepeated measures study.SettingThis study evaluated the responses of five LLMs, Google Gemini, Microsoft (Copilot), ChatGPT, Meta, and Claude artificial intelligence (AI), through a standardized set of 28 questionnaires related to NAM in cleft care.ParticipantsNone.InterventionThe accuracy of LLMs was assessed using a five-point modified Likert scale. Readability was evaluated using two validated metrics: the Flesch-Kincaid Reading Ease and Flesch-Kincaid Grade Level.Main Outcome MeasureThe primary outcome variable was the response generated by the five LLMs. Two investigators independently assessed the quality of responses from the five LLMs using a five-point modified Likert scale, with the highest score (5) indicating the highest quality.ResultsClaude AI achieved the highest mean Likert score (3.71 ± 0.53), whereas Gemini had the lowest score (3.29 ± 0.60). The highest mean readability score was observed in Meta AI (79.61 ± 37.09), while Claude AI showed significantly lower scores (47.04 ± 46.29).ConclusionAmong the five LLMs, Claude AI achieved the highest accuracy, followed by Microsoft Copilot, ChatGPT, Meta AI, and Google Gemini in responding to NAM-related queries. The responses from Claude AI were complex and harder to read, followed by ChatGPT, Copilot, Gemini, and Meta AI, with Meta AI being the most straightforward to comprehend.