Related Experiment Video
Updated: Jul 2, 2026
In Vivo Quantification of Hip Arthrokinematics during Dynamic Weight-bearing Activities using Dual Fluoroscopy
Published on: July 2, 2021
Artificial Intelligence Answering Femoroacetabular Impingement Patient Questions: Helpful Tool or Harmful Risk?
William J Ferris1, Michael G Johnston1, Harrison E Diaz1
1Department of Orthopaedic Surgery, Naval Medical Center San Diego, San Diego, CA 92134, United States.
Introduction:
As patients increasingly seek medical information online, artificial intelligence (AI) chatbots like NIPRGPT-the most widely available AI tool for Department of Defense (DOD) computer users-offer a novel resource for addressing queries about femoroacetabular impingement (FAI). To date, there have not been any studies evaluating NIPRGPT responses to orthopedic medical questions. The primary objective of this study was to evaluate the accuracy, comprehensiveness, and readability of NIPRGPT's responses to common FAI-related questions.
Materials And Methods:
Twelve frequently asked questions (FAQs) regarding FAI were selected from a curated list and posed to NIPRGPT. The accuracy and adequacy of the responses were graded by a panel of board-certified surgeons as excellent (not requiring clarification), satisfactory (requiring minimal clarification), satisfactory (requiring moderate clarification), or unsatisfactory (requiring substantial clarification). Additionally, readability was assessed using the Flesch-Kincaid readability score.
Results:
Of the 12 responses, four (33.3%) were excellent, requiring no clarification, seven (58.3%) were satisfactory, requiring minimal clarification, and one (8.3%) was satisfactory, requiring moderate clarification. No responses were deemed unsatisfactory. The average quality score was 3.38/4.0. However, the average Flesch-Kincaid readability score was a 19.6 Grade Level, indicating a reading level suited for postgraduate or specialized academic backgrounds. Interobserver agreement was low, with a Krippendorff's alpha of 0.046.
Conclusions:
NIPRGPT provides answers to FAQs about FAI that are generally accurate and reliable. However, the responses are generated at a complexity level far exceeding the recommended reading level for patient education. While a potentially useful adjunct in military healthcare settings where access may be limited, clinicians must be aware of the high literacy demand placed on patients using this tool.
