Can Artificial Intelligence Educate Patients? Comparative Analysis of ChatGPT and DeepSeek Models in Meniscus...

Area of Science:

• Orthopedics
• Medical Informatics
• Artificial Intelligence

Background:

• Meniscus injuries are common knee conditions requiring effective patient education for treatment adherence and rehabilitation.
• Artificial intelligence (AI)-based large language models (LLMs) are increasingly utilized in healthcare settings.
• Patient education is crucial for managing knee joint conditions like meniscus injuries.

Purpose of the Study:

• To compare the quality and readability of AI-generated responses to patient questions about meniscus injuries.
• To evaluate ChatGPT-5 and DeepSeek R1 for their effectiveness in patient education regarding meniscus injuries.

Main Methods:

• Twelve frequently asked questions about meniscus injuries were posed to ChatGPT-5 and DeepSeek R1.
• Responses were assessed by orthopedic surgeons for accuracy, clarity, comprehensiveness, and consistency using a rating system and Likert scale.
• Readability was measured using Flesch-Kincaid Reading Ease Score (FRES) and Grade Level (FKGL).

Main Results:

• DeepSeek R1 significantly outperformed ChatGPT-5 in overall response quality (p=0.017) and comprehensiveness (p=0.005).
• No significant differences were found in accuracy, clarity, or consistency between the two AI models.
• Both AI models generated content readable at a high-school level, with comparable readability scores.

Conclusions:

• Both ChatGPT and DeepSeek show potential as tools for patient education on meniscus injuries.
• DeepSeek R1 exhibited higher overall content quality, while both models provided accessible information.
• Further improvements in clarity and accessibility are needed for AI-generated patient education materials.