Evaluating Artificial Intelligence Conversational Platforms for Parental Queries on Antenatal Hydronephrosis: A...

Area of Science:

• Pediatric Surgery
• Medical Informatics
• Artificial Intelligence in Healthcare

Background:

• Antenatal hydronephrosis is the most common fetal anomaly. Parents frequently seek online information, including from AI platforms.
• The reliability of AI responses for pediatric surgical conditions is not well-established.

Purpose of the Study:

• To compare the quality of responses from ChatGPT, Gemini, and Claude regarding antenatal hydronephrosis.
• To assess AI-generated information for parents facing this common fetal anomaly.

Main Methods:

• Standardized questions on antenatal hydronephrosis were developed by pediatric surgeons.
• Three AI platforms (ChatGPT, Gemini, Claude) responded to questions posed by a non-medical individual.
• Pediatric surgeons evaluated responses for veracity, clarity, comprehensiveness, tone, and safety using Likert scales.

Main Results:

• AI platforms generated broadly comparable responses, with mean scores between 3.53 and 4.13 (out of 5).
• No statistically significant differences were found between ChatGPT, Gemini, and Claude.
• Inter-rater reliability among expert reviewers was poor to moderate (ICC 0.00–0.60), indicating variability in assessment.

Conclusions:

• AI conversational platforms offer similar outputs for antenatal hydronephrosis queries.
• Responses are generally clear, reassuring, and safe, but expert interpretation varies.
• AI should be viewed as a supplementary tool, not a replacement for professional medical counseling.