Foot and Ankle Patient Education Materials and Artificial Intelligence Chatbots: A Comparative Analysis

Area of Science:

• Orthopaedic patient education
• Artificial intelligence in healthcare

Background:

• Patient education materials for foot and ankle conditions are crucial for informed decision-making.
• The American Orthopaedic Foot and Ankle Society (AOFAS) provides a recommended resource, FootCareMD.org.

Purpose of the Study:

• To conduct a comparative analysis of patient education materials generated by AI chatbots versus the AOFAS-recommended website, FootCareMD.org.
• To evaluate the accuracy of AI-generated content for common foot and ankle conditions.

Main Methods:

• ChatGPT, Google Bard, and Bing AI were utilized to generate educational content on 10 prevalent foot and ankle conditions.
• AI-generated content was compared against FootCareMD.org for accuracy across various information categories, including background, symptoms, causes, diagnosis, treatments, surgical options, recovery, and risks/prevention.

Main Results:

• AI language models demonstrated consistently low accuracy rates, scoring below 60% when compared to FootCareMD.org.
• ChatGPT achieved an average accuracy of 46.2%, Google Bard 36.5%, and Bing AI 28.0% for key content across all analyzed conditions (P < .005).

Conclusions:

• AI-generated patient education for foot and ankle conditions exhibits limited content accuracy.
• Healthcare professionals and patients should exercise caution when using AI chatbots for medical information.