Rating certainty when the target threshold is the null and the point estimate is close to the null

Area of Science:

• Medical research methodology
• Evidence-based medicine

Background:

• Current methods for rating evidence certainty face challenges when point estimates are near the null effect.
• Narrow confidence intervals (CIs) near the null can lead to inappropriate downrating for imprecision.

Purpose of the Study:

• To address challenges in rating evidence certainty when point estimates are close to the null effect.
• To propose a revised approach for rating certainty of evidence.

Main Methods:

• Revising the target of certainty rating from a non-zero effect to a 'little or no effect' range.
• Estimating ranges for minimal important difference (MID) for benefit and harm.
• Adjusting certainty ratings based on whether the point estimate and CI fall within the 'little or no effect' range.

Main Results:

• The proposed method allows for appropriate rating down for imprecision only when the CI includes an important effect.
• Confidence in evidence certainty can be revised when the point estimate suggests an effect below the smallest plausible MID.
• The approach prevents downrating for imprecision when a narrow CI crosses the null.

Conclusions:

• The revised method provides a more accurate assessment of evidence certainty, especially when effects are small.
• This approach enhances the utility of the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) system.
• Optimal target selection for certainty rating is facilitated by considering the minimal important difference.