Automatic Conversion of NICE Guidelines to an Executable Computational Model Using Large Language Models

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Guideline Development

Background:

• National Institute for Health and Care Excellence (NICE) guidelines offer evidence-based clinical recommendations but exist as unstructured text.
• Current methods for converting guidelines into computable formats are disease-specific, labor-intensive, and lack scalability.
• Advances in large language models (LLMs) present an opportunity to automate the translation of clinical guidelines.

Purpose of the Study:

• To develop and evaluate an end-to-end approach for automatically converting textual clinical guidelines into executable models.
• To enable the generation of explainable, patient-specific recommendations from digitized guidelines.
• To ensure transparency and modifiability throughout the guideline conversion process.

Main Methods:

• An LLM-based, stepwise transformation process with in-context examples was employed.
• The approach was customized and applied to NICE guidelines for pancreatic and lung cancer.
• Human expert review assessed rule alignment, and an executable model was evaluated on pancreatic cancer patient vignettes.

Main Results:

• Expert review confirmed strong alignment between natural language guidelines and generated executable models, with high accuracy in recommendation translation.
• Discrepancies were primarily minor omissions, with rare instances of incorrect logic or hallucinations.
• The executable models achieved an F1 score of 82.5% when generating patient-specific recommendations for pancreatic cancer vignettes.

Conclusions:

• LLMs can effectively automate the transformation of unstructured NICE guidelines into interpretable and executable models.
• The generated models maintain guideline structure, facilitate transparent inspection and modification, and enable patient-specific recommendation generation.
• This automated approach demonstrates the feasibility of creating scalable, computable clinical guidelines.