Evaluating the performance of ChatGPT in responding to myotonic dystrophy type 1 patient inquiries: a specialist...

Advancements in artificial intelligence have led to the widespread use of large language models such as ChatGPT in healthcare communication. Myotonic dystrophy type 1, a chronic and multisystemic neuromuscular disorder, poses significant challenges for patients in understanding disease progression, symptom management, and future planning. Given the scarcity of specialized resources for rare diseases, AI-based tools may offer valuable support by delivering accessible and relevant information. This study aimed to evaluate ChatGPT-4o's performance in addressing patient-centered questions about myotonic dystrophy type 1.

This descriptive methodological study utilized ten frequently asked patient-style questions concerning myotonic dystrophy type 1, posed to ChatGPT-4o in Turkish using a standardized zero-shot prompting approach. A total of 53 specialist physicians in Physical Medicine and Rehabilitation and Neurology assessed the responses using a structured rubric across five domains: accuracy, currency, comprehensiveness, usefulness, and understandability. Each response was scored from 1 to 5.

ChatGPT-4o achieved high specialist physicians' satisfaction across all domains, with over 80% of responses rated ≥ 4. Accuracy received the highest score (85.7%), followed by usefulness and comprehensiveness (above 82%). Currency and understandability were rated slightly lower (81.9% and 81.5%, respectively).

ChatGPT-4o generated responses that were generally accurate, relevant, and understandable, demonstrating promise as an accessible resource for rare disease contexts such as myotonic dystrophy type 1. It also showed strong potential as a supportive informational tool for specialist physicians. Ongoing model refinement and integration with up-to-date clinical data are essential to optimize its performance and ensure its safe, equitable use in healthcare communication.