How updating textual clinical practice guidelines impacts clinical decision support systems: a case study with...

Guideline-based clinical decision support systems (CDSSs) can be effective in increasing physician compliance with recommendations. However, the ever growing pace at which medical knowledge is produced requires that clinical practice guidelines (CPGs) be updated regularly. It is therefore mandatory that CDSSs be revised accordingly. The French Association for Urology publishes CPGs on bladder cancer management every 2 years. We studied the impact of the 2004 revision of these guidelines, with respect to the 2002 version with a CDSS, UroDoc. We proposed a typology of knowledge base modifications resulting from the update of CPGs making the difference between practice, clinical conditions and recommendations refinement as opposed to new practice and new recommendations. The number of formalized recommendations increased from 577 in 2002 to 1,081 in 2004. We evaluated the two versions of UroDoc on a randomized sample of patient records. A single new practice that modifies a decision taken in 49% of all recorded decisions leads to a fall from 67% to 46% of the compliance rate of decisions.