Assessing the accuracy and reliability of ChatGPT's medical responses about thyroid cancer

ChatGPT has the potential to offer patient-friendly support. Thyroid carcinoma has become increasingly prevalent in recent years. This study aimed to assess ChatGPT's accuracy and adequacy in answering questions about information, management, and emotional support related to thyroid cancer.

We conducted a three-step study. In the first step, ChatGPT responded to 30 questions about thyroid cancer. In the second step, we presented three different cases of thyroid cancer patients and asked ChatGPT about their diagnosis, treatment management, and follow-up. In the third step, we inquired about emotional support for patients and their families. Three expert endocrinologists graded these responses according to ATA guidelines.

We showed that ChatGPT regurgitated extensive knowledge of thyroid cancer (76.66% correct), but only small proportions (6.66%) were labeled as mixed with correct and incorrect/outdated data. However, it was inadequate in the evaluation of clinical cases of thyroid cancer. It mentioned treatment and follow-up recommendations in a general framework, not patient-specific. Also, it provided practical and multifaceted emotional support advice to patients and caregivers regarding the next steps and adjusting to a new diagnosis.

Our study is the first to evaluate the competence and reliability of ChatGPT in thyroid cancer. Although ChatGPT is moderately competent in obtaining information about thyroid cancer, it has not yet been determined to be sufficiently competent and reliable in case management. It has been found effective in guiding patients and their relatives regarding emotional support.