Assessing ChatGPT on Male HPV Disorders: Repeated Assessments of Urological and Dermatological Accuracy Using...

Human papillomavirus (HPV) is a prevalent viral infection that poses considerable health risks to males, including genital warts and various cancers. The epidemiology of HPV in the male population requires novel approaches for effective patient education. This study investigated the efficacy of Chat Generative Pre-trained Transformer-4 (ChatGPT-4), a sophisticated artificial intelligence model, in disseminating accurate information about HPV-induced genital warts in men.

A meticulously designed questionnaire comprising 240 items, spanning multiple-choice and true/false formats, was administered to ChatGPT-4 from October 2024 to January 2025. The questionnaire covered HPV transmission, clinical manifestations, diagnostic approaches, treatment options and preventive strategies. The responses were evaluated for accuracy and completeness using Likert scales, with repeated questioning sessions to assess the potential for response improvement over time.

ChatGPT-4 initially provided accurate and/or highly rated responses for 191 out of 240 questions. Subsequent rounds of questioning for inaccurately answered or low-scoring questions showed a progressive improvement, with 14 and 15 questions answered correctly or receiving higher accuracy scores in the second and third rounds, respectively. The overall accuracy rate was 79.60%, with specific subgroup analyses revealing strengths in general information and protection but challenges in questions about HPV-induced genital warts in men.

ChatGPT-4 demonstrates preliminary medical value and potential for patient education in urology. However, variable subgroup accuracy suggests that further optimisation and validation are needed before clinical deployment. Its ability to accurately address a wide range of queries, combined with the potential for an improvement in response accuracy over time, highlights its value in enhancing health communication.