Let's chat about cervical cancer: Assessing the accuracy of ChatGPT responses to cervical cancer questions

To quantify the accuracy of ChatGPT in answering commonly asked questions pertaining to cervical cancer prevention, diagnosis, treatment, and survivorship/quality-of-life (QOL).

ChatGPT was queried with 64 questions adapted from professional society websites and the authors' clinical experiences. The answers were scored by two attending Gynecologic Oncologists according to the following scale: 1) correct and comprehensive, 2) correct but not comprehensive, 3) some correct, some incorrect, and 4) completely incorrect. Scoring discrepancies were resolved by additional reviewers as needed. The proportion of responses earning each score were calculated overall and within each question category.

ChatGPT provided correct and comprehensive answers to 34 (53.1%) questions, correct but not comprehensive answers to 19 (29.7%) questions, partially incorrect answers to 10 (15.6%) questions, and completely incorrect answers to 1 (1.6%) question. Prevention and survivorship/QOL had the highest proportion of "correct" scores (scores of 1 or 2) at 22/24 (91.7%) and 15/16 (93.8%), respectively. ChatGPT performed less well in the treatment category, with 15/21 (71.4%) correct scores. It performed the worst in the diagnosis category with only 1/3 (33.3%) correct scores.

ChatGPT accurately answers questions about cervical cancer prevention, survivorship, and QOL. It performs less accurately for cervical cancer diagnosis and treatment. Further development of this immensely popular large language model should include physician input before it can be utilized as a tool for Gynecologists or recommended as a patient resource for information on cervical cancer diagnosis and treatment.