AI's deep dive into complex pediatric inguinal hernia issues: a challenge to traditional guidelines?

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Pediatric Surgery

Background:

• Large language models (LLMs) offer potential for clinical decision support.
• Controversial topics in pediatric surgery require adherence to established guidelines.

Purpose of the Study:

• To evaluate the accuracy and reliability of ChatGPT 4.0 responses on pediatric inguinal hernia surgery issues.
• To compare AI-generated answers with European Association of Pediatric Surgeons (EUPSA) guidelines.

Main Methods:

• Six controversial EUPSA topics were input into ChatGPT 4.0, generating two responses per topic.
• Responses were compared against systematic reviews and EUPSA guidelines using content analysis and expert evaluation.
• TF-IDF model assessed response stability; expert scoring evaluated quality, reliability, and applicability.

Main Results:

• ChatGPT 4.0 responses largely aligned with EUPSA guidelines, with some noted discrepancies.
• Average scores: Quality 3.33, Reliability 2.75, Applicability 3.46 (out of 5).
• Average similarity between AI responses was 0.72 (out of 1), indicating reasonable consistency.

Conclusions:

• ChatGPT demonstrates potential for answering clinical queries but has limitations.
• Further refinement and integration with reliable data sources are necessary for optimal clinical practice.
• AI tools should supplement, not replace, expert judgment and evidence-based guidelines.