Acute Achilles tendon rupture: how well can artificial intelligence chatbots answer patient inquiries?

Area of Science:

• Orthopaedic Surgery
• Artificial Intelligence in Medicine
• Medical Information Dissemination

Background:

• Artificial intelligence (AI) chatbots are increasingly used by patients for medical information.
• Optimal treatment for acute Achilles tendon ruptures (AATR) is debated due to varied approaches.
• Patients seek AI-driven answers for medical queries, necessitating evaluation of AI response accuracy.

Purpose of the Study:

• To assess the adequacy of responses from popular AI engines to frequently asked questions regarding AATR.
• To compare the performance of different AI chatbots in providing AATR information.

Main Methods:

• Ten common AATR management questions were posed to three AI engines: ChatGPT, Google Gemini, and Microsoft Copilot.
• Responses were evaluated by four board-certified orthopaedic surgeons using a four-point scale.
• Kruskal-Wallis tests and post-hoc analyses compared AI response scores.

Main Results:

• All three AI engines provided comparable answers for 70% of the AATR questions.
• Significant differences in response quality were observed for three questions (p < 0.05).
• Microsoft Copilot received lower scores compared to Google Gemini and ChatGPT for specific questions.

Conclusions:

• AI chatbots can adequately address concise patient queries on AATR diagnosis and management.
• The analyzed AI chatbots provided largely uniform and satisfactory responses.
• Advancing AI engines show promise as valuable tools for patient education in orthopaedics.