Evaluating the accuracy and comprehensiveness of GPT-4 as a patient information tool for pectus deformities

BackgroundPectus excavatum and carinatum are the most common anterior chest wall deformities in children and adolescents. With growing awareness and easy access to online resources, patients increasingly seek information about these conditions from artificial intelligence platforms.ObjectiveThe aim of this study is to evaluate the artificial intelligence model Chat Generative Pre-trained Transformer (ChatGPT) by analyzing its responses to frequently asked questions about pectus deformities and assessing their accuracy and comprehensiveness.MethodsWebsites frequently used by patients with pectus and their families, as well as quality-of-life questionnaires, have been examined by the authors, and a total of 99 questions have been collected. ChatGPT's responses were evaluated by a physiatrist and a pediatric surgeon using a 4-point grading system: 1) correct and comprehensive, 2) correct but not comprehensive, 3) partially correct and partially incorrect, 4) completely incorrect.Results51 questions (51.5%) received correct and comprehensive answers, 40 questions (40.4%) received correct but not comprehensive answers, 8 questions (8.1%) received partially incorrect answers, and none received completely incorrect answers. ChatGPT was least successful in the pectus carinatum treatment and follow-up category, answering 19 questions with 21.1% correct, but not comprehensively.ConclusionsChatGPT's responses about pectus deformities have been found highly accurate, and more than half of the responses are comprehensive. It can be a useful and fast source of information for basic patient education. However, every patient is unique, and treatment and follow-up require a holistic evaluation. This point seems to be a limitation of ChatGPT and underscores the indispensable role of specialized medical consultants.