A custom ChatGPT can accurately answer questions from an international expert osteotomy consensus statement

Area of Science:

• Orthopedic surgery
• Artificial intelligence in medicine
• Medical education

Background:

• High tibial osteotomy (HTO) is a common procedure for managing knee osteoarthritis.
• Access to accurate, evidence-based information on HTO is crucial for surgeons.
• Existing AI models may lack specialized orthopedic knowledge.

Purpose of the Study:

• To evaluate the accuracy of a custom-trained ChatGPT in answering questions about high tibial osteotomies (HTO).
• To use an international expert osteotomy consensus statement as the knowledge base for the AI.
• To assess the AI's performance in providing relevant and complete information.

Main Methods:

• A custom ChatGPT was trained using the European Society of Sports Traumatology, Knee Surgery and Arthroscopy (ESSKA) osteotomy consensus statement.
• The AI was tested with 10 questions: 5 identical to the consensus and 5 common HTO queries.
• Three knee surgeons evaluated responses based on accuracy, relevance, clarity, completeness, and adherence to consensus using a Likert scale.

Main Results:

• The AI achieved high mean scores for accuracy (2.5/3), relevance (2.9/3), and clarity (2.9/3) with good inter-rater reliability.
• Completeness scores averaged 2.6/3 (moderate reliability), and adherence to consensus was 2.5/3 (excellent reliability).
• Adherence to the consensus document was lower for general HTO questions compared to directly extracted ones.

Conclusions:

• Custom-trained ChatGPT demonstrates high accuracy in answering HTO-related questions based on expert consensus.
• This AI tool can effectively provide evidence-based answers, supporting surgeons in clinical decision-making.
• The approach is potentially transferable to other surgical specialties and consensus documents.