Examining the effectiveness of ChatGPT responses to frequently asked questions by individuals with postural disorders

The aim of the study was to evaluate the quality and readability of ChatGPT responses to frequently asked questions by individuals with posture disorder. Providing reliable and evidence-based information about posture disorders is vital for individuals to be correctly informed.

The 10 most frequently asked questions about posture disorder were selected by two researchers from a list created by ChatGPT. The questions were transmitted to ChatGPT 4.0, and the initial responses were recorded without further follow-up questions. The quality of the responses was then assessed by five independent experts (three physiotherapists, one physical therapy and rehabilitation specialist, and one orthopedics and traumatology specialist) with a four-grade evaluation system. Readability levels were analyzed with the Flesch-Kincaid Grade Level through WordCalc software. Statistical analysis was performed using Statistical Package for the Social Sciences v29.0, and intraclass correlation coefficients were used to measure inter-rater reliability.

Following a thorough evaluation of the 10 responses received, six were rated as "Excellent responses requiring no explanation," while a further four were designated as "Satisfactory responses requiring minimal explanation." The median quality score of the responses was high, indicating good alignment with current evidence-based practice. The average readability level of the responses was determined to be 8.4. Inter-rater reliability was good, with an intraclass correlation coefficients value of 0.756.

ChatGPT provides relatively coherent and generally readable answers to frequently asked questions about posture disorders, with most needing minimal explanation. While promising as a resource to meet the information needs of people with posture disorders, further improvements are needed to align it with personalized health needs.