Lessons Learned From AI-Assisted Guideline Generation in Parastomal Hernia Repair

Large language models (LLMs) can analyse scientific literature and draft medical recommendations, but their role in formal clinical guideline development is unclear.

To evaluate whether a publicly available GPT-based LLM can generate coherent, GRADE-based guidelines for parastomal hernia management from a predefined evidence base, and to compare these with the 2017 European Hernia Society (EHS) guidelines. A secondary aim was to explore implications for academic publishing and scientific authorship.

The 2017 EHS parastomal hernia guidelines (Antoniou et al.) were used as the reference framework. Within a closed session, the model was instructed to apply AGREE II and GRADE principles to 52 full-text clinical papers mirroring the original EHS reference set, and to formulate recommendations for nine key clinical questions (KQs). For each KQ, the model defined PICO, summarized the evidence, rated certainty, and stated direction and strength of recommendation. AI-derived guidance was then systematically compared with EHS statements. Divergences were classified as interpretative, threshold-based (handling of low-certainty evidence), or evidence-weighting.

AI-generated recommendations showed full or near-full alignment with EHS guidance in most domains, including diagnosis, prophylactic mesh for permanent end colostomy, rejection of suture-only repair, preference for non-keyhole laparoscopic repair, and favouring synthetic over biologic meshes. Differences arose primarily where evidence was very low quality: the model issued cautious, conditional recommendations (e.g., watchful waiting in asymptomatic hernias, consideration of laparoscopy in suitable patients, preference for retromuscular synthetic mesh and avoidance of cross-linked collagen onlay), whereas EHS opted for no recommendation.

Within a closed evidence base, a GPT-based model can reproduce the logic and structure of expert guideline development with high fidelity. Discrepancies mainly reflect different thresholds for acting on low-certainty evidence, supporting a complementary role for AI as a structured methodological and drafting assistant rather than a replacement for human consensus.