Comparing Accuracy and Completeness of Google Search Versus ChatGPT-4 Responses to Questions Patients Have Regarding...

Patients often use Google as a source of quick medical information, although the accuracy and clarity of search results can vary. ChatGPT has emerged as an alternative tool capable of providing conversational and potentially more reliable medical information. This study compares the readability, accuracy, and completeness of responses generated by ChatGPT with those obtained using Google for common patient questions regarding craniosynostosis and cleft palate.

The terms "Craniosynostosis" and "Cleft Palate" were entered into Google, and the top 10 associated questions for each-identified using the "People Also Ask" tool-were recorded. Each question was then entered into both Google and ChatGPT, and the responses from each were recorded. The ease of readability for each response was determined by the Flesch-Kincaid instrument. Blinded reviewers evaluated accuracy and completeness using a 3-point scale (1 = fully incorrect, 2 = partially incorrect, 3 = correct). Reviewer scores were averaged, and comparisons between platforms were evaluated using t tests.

A total of 20 questions yielded 40 unique responses. For cleft palate queries, Google responses had significantly lower reading levels than ChatGPT (9.95 vs 13.22, P  = 0.006). No significant difference in readability was observed for craniosynostosis responses (14.66 vs 14.73, P  = 0.467). Across all questions, ChatGPT responses were significantly more complete (2.60 vs 1.86, P  < 0.0001) and more accurate (2.78 vs 2.09, P  < 0.0001) than Google responses. These differences persisted when each condition was analyzed separately.

ChatGPT provides more accurate and comprehensive information than Google for common patient questions about craniosynostosis without sacrificing readability. Patients can use this information to inform their future searches in order to obtain the most accurate information about their diagnoses. Further studies evaluating the information learned by patients from both search engines can help clinicians guide patients toward resources that best fit their individual care.