A Comparative Analysis of Pulmonary and Critical Care Medicine Guideline Development Methodologies

The Institute of Medicine (IOM) standards for guideline development have had unintended negative consequences. A more efficient approach is desirable.

To determine whether a modified Delphi process early during guideline development discriminates recommendations that should be informed by a systematic review from those that can be based upon expert consensus.

The same questions addressed by IOM-compliant pulmonary or critical care guidelines were addressed by expert panels using a modified Delphi process, termed the Convergence of Opinion on Recommendations and Evidence (CORE) process. The resulting recommendations were compared. Concordance of the course of action, strength of recommendation, and quality of evidence, as well as the duration of recommendation development, were measured.

When 50% agreement was required to make a recommendation, all questions yielded recommendations, and the recommended courses of action were 89.6% concordant. When 70% agreement was required, 17.9% of questions did not yield recommendations, but for those that did, the recommended courses of action were 98.2% concordant. The time to completion was shorter for the CORE process (median, 19.3 vs. 1,309.0 d; P = 0.0002).

We propose the CORE process as an early step in guideline creation. Questions for which 70% agreement on a recommendation cannot be achieved should go through an IOM-compliant process; however, questions for which 70% agreement on a recommendation can be achieved can be accepted, avoiding a lengthy systematic review.