From Guidelines to Code: Formalizing STOPP/START Criteria Using LLMs and RAG for Clinical Decision Support

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Computational Pharmacology

Background:

• Polypharmacy in elderly patients presents significant challenges for medication management.
• The STOPP/START v3 criteria offer a framework for optimizing therapy but are difficult to implement in software.
• Formalizing complex clinical guidelines into executable code is a critical step for clinical decision support systems.

Purpose of the Study:

• To automate the formalization of STOPP/START v3 criteria using large language models (LLMs).
• To leverage Retrieval-Augmented Generation (RAG) for enhanced accuracy in rule formalization.
• To generate executable code from medical guidelines for improved prescribing software.

Main Methods:

• Utilized LLMs (DeepSeek, GPT-4o-mini) for entity extraction and code mapping (ICD-10, LOINC, ATC).
• Employed Retrieval-Augmented Generation (RAG) to improve the accuracy of LLM-based rule formalization.
• Developed a process to generate executable Python code from formalized clinical rules.

Main Results:

• Achieved high F1-scores for entity mapping: 0.90 (drug), 0.92 (disease), and 1.0 (observation).
• Demonstrated perfect results in medical entity extraction and code logic consistency.
• Successfully transformed complex clinical rules into accurate, executable code.

Conclusions:

• LLMs, particularly with RAG, can effectively automate the formalization of medical guidelines.
• This automation facilitates the integration of clinical decision support tools into prescribing software.
• The approach shows significant promise for advancing clinical decision support and formalizing medical rules.