An Artificial Intelligence Application for Standardized Expert-Level Interpretation of Vascular Clinical Practice...

Clinical practice guidelines (CPGs) are often complex and subject to the reader's interpretation. The aim was to develop and validate an artificial intelligence (AI)-driven application for standardized interpretation of CPGs. The application was named "VascLink-AI," reflecting the clinical focus and traceable nature of the tool.

A comparative study to benchmark AI performance against established clinical standards. The 2017 European Society of Cardiology/European Society for Vascular Surgery Guidelines on the Diagnosis and Treatment of Peripheral Arterial Diseases were vectorised into a knowledge graph. A large language model was locked to this graph, returning structured, citation-anchored answers. Performance was tested on 41 committee-answered vignettes. The app's answers and the committee's answers were scored for accuracy, completeness, clarity, relevance, adaptability, and evidence justification by an automated evaluator (GPT-4o) and 2 independent human experts. Inter-rater reliability and consensus scores were compared. A noninferiority analysis was performed.

The application achieved high alignment with the expert intent, with composite scores favoring the app for both raters: automated 90.5% versus 77.8%, P < 0.001; human consensus 94.2% versus 86.2%, P < 0.001. Reliability analysis revealed an "agreement gap"; human raters' agreement on committee answers was negligible (Kappa = -0.018), but reached a moderate level on AI answers (Kappa = 0.450). The application was found to be noninferior to the experts across all 12 evaluated metrics (all P < 0.001).

This study demonstrates how AI can grow from a general research aid into a traceable, guideline-restricted framework capable of delivering advice that matches human expert interpretation of societal guidelines.