Performance of artificial intelligence chatbots compared with young academic urologists in reconstructive urology.

Area of Science:

• Urology
• Artificial Intelligence
• Medical Education

Background:

• The application of artificial intelligence (AI) in surgical fields is expanding, yet its specific utility in reconstructive urology requires further investigation.
• Existing research has not sufficiently explored the theoretical knowledge capabilities of AI platforms within reconstructive urology.

Purpose of the Study:

• To assess the theoretical knowledge of AI platforms in reconstructive urology.
• To compare the performance of AI platforms against experts from the Young Academic Urologists (YAU) group.

Main Methods:

• A cross-sectional comparative study involving 11 YAU experts and 4 AI platforms.
• Thirty-one multiple-choice questions from a leading urology review text were administered.
• Performance was evaluated based on total and thematic scores.

Main Results:

• AI platforms achieved a mean score of 15/31, statistically similar to the YAU experts' mean score of 14/31 (p=0.802).
• ChatGPT-4 scored 24/31, outperforming most human experts.
• No significant differences were found in performance across specific domains including anatomy, physiology, diagnosis, and therapeutic management.

Conclusions:

• AI platforms, exemplified by ChatGPT-4, demonstrate strong performance on theoretical multiple-choice questions in reconstructive urology.
• AI shows potential for use in urological education and answering defined theoretical queries.
• Clinical implementation requires further validation with real patient cases, as AI is not yet adequate for complex clinical reasoning.