The quality of obstetrical clinical practice guidelines promulgated by a specialty society

To review the methodologic quality of all obstetrical clinical practice guidelines developed by the Society of Obstetricians and Gynaecologists of Canada (SOGC) from 1992 to August 2001.

Three reviewers independently assessed each of 37 guidelines according to a validated and reliable 37-item appraisal tool. A mean "global" score as well as three "dimension" scores were calculated for each guideline. The appraisal tool assessed each document according to three dimensions, which related to the rigour of its development (Dimension 1), its context and content (Dimension 2), and its application (Dimension 3). A repeated-measures analysis of variance was used to derive the interclass correlation coefficient.

Mean global quality scores ranged from 8.1% to 54.0%. Only two guidelines were given a mean global score above 50% and 21 of the guidelines (56.8%) had mean global scores of less than 30%. Mean dimension scores were 19.9% for Dimension 1, relating to rigour of development, 47.3% for Dimension 2, relating to context and content, and 27.2% for Dimension 3, relating to application. The interclass correlation coefficient using a fixed-effects model was 0.72, reflecting reasonable agreement between the reviewers.

Both the mean global scores and mean dimension-specific scores for the obstetrical clinical practice guidelines were lower than optimal. We were unable to reveal statistically significant improvements in guideline quality over time given the limited number of documents, but the scores for more recently drafted guidelines appear generally higher than earlier guidelines. This finding is consistent with new editorial policy adopted by the SOGC regarding guideline development. A broader review should be conducted by the SOGC on guidelines in development and under revision.