Evaluating ChatGPT's Ability to Answer Common Patient Questions Regarding Hip Fracture

ChatGPT is an artificial intelligence chatbot software programmed for conversational applications using reinforcement learning techniques. With its growing popularity and overall versatility, it is likely that ChatGPT's applications will expand into health care especially because it relates to patients researching their injuries. The purpose of this study was to investigate ChatGPT's ability to accurately answer frequently asked questions regarding hip fractures.

Eleven frequently asked questions regarding hip fractures were posed to ChatGPT, and the responses were recorded in full. Five of these questions were determined to be high-yield based on the likelihood that a patient would ask the question to a chatbot software. The chatbot's responses were analyzed by five fellowship-trained orthopaedic trauma surgeons for their quality and accuracy using an evidence-based approach. The chatbot's answers were rated as "Excellent response requiring no clarification", "Satisfactory response requiring minimal clarification", "Satisfactory response requiring moderate clarification", or "Unsatisfactory response requiring significant clarification."

Of the five high-yield questions posed to the chatbot, no question was determined to be unsatisfactory requiring significant clarification by the authors. The remaining responses were either satisfactory requiring minimal clarification (n = 3) or satisfactory requiring moderate clarification (n = 2).

The chatbot was generally found to provide unbiased and evidence-based answers that would be clearly understood by most orthopaedic patients. These findings suggest that ChatGPT has the potential to be an effective patient education tool especially because it continues to grow and improve as a chatbot application.

Level IV study.