Debunking Myths and Patient Education About Complex Regional Pain Syndrome Using Popular Generative Artificial...

Complex Regional Pain Syndrome (CRPS) is a chronic pain condition with a complex pathophysiology, significantly affecting patients' quality of life. Effective patient education is crucial for managing CRPS, but available resources often lack readability and accessibility, leading to the persistence of myths. This study evaluates the accuracy, readability, completeness, emotional tone, and myth-debunking capabilities of AI-generated patient education materials (PEMs) compared to traditional online sources.

A comparative design was employed to assess PEMs generated by three AI chatbots (ChatGPT, Google's Gemini, and Meta's Llama) against traditional online materials from Cleveland Clinic and Mayo Clinic. Readability was evaluated using the Flesch Reading Ease, Gunning Fog Index, Flesch-Kincaid Grade Level, and SMOG Index. Accuracy and completeness were assessed by a panel of five pain experts using a Likert scale (1-5). Sentiment analysis measured emotional tone, and the AI chatbots' ability to debunk 15 common CRPS myths was compared with expert evaluations using sensitivity, specificity, and other performance metrics.

Cleveland Clinic PEMs demonstrated the highest accuracy (4.8 ± 0.40 expressed as mean ± standard deviation, significantly outperforming Gemini (3.4 ± 0.49, P = .004) and Llama (3.6 ± 0.49, P = .014). Completeness was also higher for Cleveland Clinic (4.8 ± 0.40), vs Gemini (3.8 ± 0.40, P = .031) and vs Llama (3.6 ± 0.49, P = .008). Gemini had superior readability (Flesch score 62 as compared to Traditional PEMs and other AI generated PEMs all scoring <52) but lower specificity (66.67%) in debunking myths. ChatGPT had the highest specificity (100%). The sentiment analysis showed that Gemini produced the most positive content (sentiment score 83), while the Cleveland Clinic's tone was neutral (sentiment score -8.0).

AI-generated PEMs show promise as effective tools for patient education in CRPS, offering accurate and readable content. However, traditional online resources from prestigious institutes still provide more comprehensive information. The study highlights the potential of AI in complementing traditional online educational materials, but further validation and improvements are needed to optimize their use in clinical practice.