Evaluating ChatGPT's Role in Patient Education on Distraction Osteogenesis for Limb Lengthening

Limb lengthening requires patients to understand a complex procedure, a prolonged recovery, and potential complications. This study evaluated the quality, reliability, readability, and clinical completeness of ChatGPT-3.5 (OpenAI, San Francisco, CA) responses to common patient questions about limb lengthening.

In this descriptive, cross-sectional content analysis, a purposive sample of 10 commonly encountered and clinically relevant questions was identified from the frequently asked questions pages of 10 healthcare institutions. Each question was entered separately into ChatGPT-3.5 on January 18, 2024, without follow-up prompts. Two senior authors independently evaluated each response against published literature and scored it with the Quality Criteria for Consumer Health Information (DISCERN) instrument, Journal of the American Medical Association (JAMA) benchmark criteria, and Flesch-Kincaid grade level. Outcomes were summarized descriptively, and interrater agreement was assessed with the Cohen kappa coefficient.

The mean DISCERN score was 35.5 (range: 21-42). One response was rated very poor, seven were rated poor, and two were rated fair. Every response received a JAMA benchmark score of 0. The mean Flesch-Kincaid grade level was 13.6 (range: 12.3-15.7), indicating postsecondary reading demand.

ChatGPT-3.5 provided broad introductory information, but most responses lacked transparent sources, required clinical clarification, and were too complex for general patient education. It may be useful as a clinician-reviewed adjunct but should not replace individualized counseling by a qualified healthcare professional.