Rating certainty of evidence when the target threshold is a value-based threshold and the point estimate is close to...

It is crucial, when using Grading of Recommendations Assessment, Development and Evaluation (GRADE), that authors are clear about what it is in which they are rating the certainty of evidence (ie, the target of certainty of evidence rating). This article addresses challenges GRADE users face in choosing a target of certainty rating for any value-based threshold (ie, little or no difference; an important effect; small, moderate or large difference) when the point estimate is close to that threshold.

Using an iterative process, authors generated and refined possible solutions to the identified challenges.

The challenges GRADE users face when point estimates prove close to the initially chosen value-based threshold include (1) the intuitive interpretation of the point estimate is similar whichever side of the threshold the point estimate lies on but the formal interpretation differs greatly and (2) rating down for imprecision due to the confidence interval (CI) crossing the threshold when the CI is very narrow. This paper provides four possible solutions. The first is continuing to rate certainty in an important or unimportant effect and rating down for imprecision when the CI crosses the MID. The second is to consider uncertainty around the MID, rating certainty in relation to a range of plausible MIDs (ie, a range formed by the smallest plausible MID and the largest plausible MID), and rate down for imprecision when the CI crosses either plausibility bound. The third solution is considering both the null and moderate effect thresholds, rating certainty in a trivial or small effect, and rating down for imprecision when the CI crosses either threshold. The fourth solution is using more than one of the above solutions and presenting two or more certainty of evidence ratings.

GRADE users should note these challenges and apply their chosen solution to transparently determine the target of their certainty rating.

When point estimates prove to be close to the initially chosen value-based threshold, to decide on what it is in which GRADE users rate their certainty (ie, the target of certainty rating), GRADE users face challenges. This paper provides four possible solutions related to the minimal important difference (MID) that are also applicable to other value-based thresholds. (1) Continuing to rate certainty in an important or unimportant effect and rate down for imprecision when the CI crosses the MID; (2) Considering a plausible range of MIDs, rating certainty in an effect that is close to the MID, and rating down for imprecision when the CI crosses either or both plausibility bounds; (3) Considering both the null and moderate effect thresholds, rating certainty in a trivial or small effect, and rating down for imprecision when the CI crosses either threshold; and (4) presenting more than one certainty of evidence rating. GRADE users may note these challenges and consider applying the solutions to transparently determine the target of their certainty rating and to make certainty assessments accordingly.