Strong recommendations from low certainty evidence: a cross-sectional analysis of a suite of national guidelines

Clinical guidelines should be based on a thorough evaluation of the evidence and generally include a rating of the quality of evidence and assign a strength to recommendations. Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance warns against making strong recommendations when the certainty of the evidence is low or very low, but has identified five paradigmatic situations (e.g. life-threatening situations) where this may be justified.

We aimed to characterize the strength of recommendations and certainty of the evidence in Irish National Clinical Guidelines using the GRADE approach.

All National Clinical Guidelines from the National Clinical Effectiveness Committee (NCEC) website using the GRADE approach (fully or partially) were included. All recommendations and their corresponding certainty of the evidence, strength of recommendations and justifications were extracted. Authors classified instances of strong recommendations with low certainty evidence (referred to as discordant recommendations) into one of the five paradigmatic situations. Descriptive statistics were calculated.

From the 29 NCEC Clinical Guidelines available at the time of analysis, we identified 8 guidelines using GRADE with a total of 240 recommendations; 38 recommendations did not use the GRADE approach and were excluded. Half of the included guidelines focused on emergency situations. In the final dataset of 202 recommendations, 151 (74.7%) were classified as strong and 51 (25.3%) as conditional. Of the 151 strong recommendations, 55 (36.4%) were supported by high or moderate certainty evidence and 96 (63.6%) by low or very low certainty evidence and were considered discordant. Of these 96 discordant recommendations, 55 (73.7%) were consistent with one of the five paradigmatic situations. However, none were specifically described as such within the guidelines.

The proportion of discordant recommendations identified in this analysis was higher than some previous international studies (range of all strong recommendations being discordant 30-50%), but similar to other guidelines focused on emergency situations. The majority of discordant recommendations could be mapped to one of the five situations, but no National Clinical Guideline explicitly referenced this. Guideline developers require further guidance to enable greater transparency in the reporting of the reasons for discordant recommendations.