Evaluating AI-Based Chatbots for Exercise and Rehabilitation Strategies in Patients With Rheumatoid Arthritis:...

Area of Science:

• Artificial Intelligence in Healthcare
• Rheumatoid Arthritis Management
• Digital Health Tools

Background:

• Artificial intelligence (AI) shows promise in healthcare efficiency, but its application in exercise and rehabilitation is not well-defined.
• Patients with rheumatoid arthritis (RA) require tailored exercise and rehabilitation strategies for optimal management.

Purpose of the Study:

• To evaluate the quality, reliability, accuracy, and readability of three large language models (LLMs)—ChatGPT-5, DeepSeek-R1, and Gemini 2.5.
• To assess LLM responses to patient-specific questions on exercise and rehabilitation for rheumatoid arthritis (RA).

Main Methods:

• A cross-sectional comparative study design was employed.
• A structured assessment framework evaluated LLM responses to RA exercise and rehabilitation questions across five domains (exercise, hand function, joint protection, breathing, general).
• Information quality (modified DISCERN), reliability (Reliability Score), accuracy (Likert scale), and readability (Flesch Reading Ease) were measured.

Main Results:

• DeepSeek-R1 and ChatGPT-5 demonstrated superior quality, reliability, accuracy, and readability compared to Gemini 2.5.
• Both DeepSeek-R1 and ChatGPT-5 consistently outperformed Gemini 2.5 in exercise and hand function domains.
• Despite higher scores, all LLMs produced responses classified as difficult to read (mean Flesch Reading Ease: DeepSeek-R1=50.20, ChatGPT-5=46.66, Gemini 2.5=37.33).

Conclusions:

• DeepSeek-R1 and ChatGPT-5 offer more accurate and reliable RA exercise information than Gemini 2.5.
• The complex language used by all evaluated LLMs may hinder accessibility for patients with low health literacy.
• Professional supervision is essential for integrating LLM-generated exercise plans into RA patient care.