Related Experiment Video
Updated: Jul 31, 2025
Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
GRADE concept 4: rating the certainty of evidence when study interventions or comparators differ from PICO targets
Marius Goldkuhle1, Gordon H Guyatt2, Nina Kreuzberger1
1Evidence-based Medicine, Department I of Internal Medicine, Faculty of Medicine and University Hospital Cologne, University of Cologne, Kerpener Str. 62, 50937 Cologne, Germany.
Objectives:
This Grading of Recommendations Assessment, Development and Evaluation (GRADE) concept article offers systematic reviewers, guideline authors, and other users of evidence assistance in addressing randomized trial situations in which interventions or comparators differ from those in the target people, interventions, comparators, and outcomes. To clarify what GRADE considers under indirectness of interventions and comparators, we focus on a particular example: when comparator arm participants receive some or all aspects of the intervention management strategy (treatment switching).
Study Design And Setting:
An interdisciplinary panel of the GRADE working group members developed this concept article through an iterative review of examples in multiple teleconferences, small group sessions, and e-mail correspondence. After presentation at a GRADE working group meeting in November 2022, attendees approved the final concept paper, which we support with examples from systematic reviews and individual trials.
Results:
In the presence of safeguards against risk of bias, trials provide unbiased estimates of the effect of an intervention on the people as enrolled, the interventions as implemented, the comparators as implemented, and the outcomes as measured. Within the GRADE framework, differences in the people, interventions, comparators, and outcomes elements between the review or guideline recommendation targets and the trials as implemented constitute issues of indirectness. The intervention or comparator group management strategy as implemented, when it differs from the target comparator, constitutes one potential source of indirectness: Indirectness of interventions and comparators-comparator group receipt of the intervention constitutes a specific subcategory of said indirectness. The proportion of comparator arm participants that received the intervention and the apparent magnitude of effect bear on whether one should rate down, and if one does, to what extent.
Conclusion:
Treatment switching and other differences between review or guideline recommendation target interventions and comparators vs. interventions and comparators as implemented in otherwise relevant trials are best considered issues of indirectness.
Insights
This study clarifies how the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework addresses indirectness in systematic reviews. It focuses on treatment switching in comparator groups to ensure evidence quality.
Area of Science:
- Evidence-based medicine
- Systematic reviews and meta-analysis
- Clinical guideline development
Background:
- Systematic reviews and guidelines rely on randomized trials, but trial conditions may differ from target populations, interventions, or comparators.
- Indirectness arises when elements like interventions, comparators, or outcomes in trials do not align with the review's focus.
- Treatment switching, where comparator groups receive the intervention, is a key challenge in assessing indirectness.
Purpose of the Study:
- To provide clarity on how the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework handles indirectness in systematic reviews.
- To define and illustrate indirectness, particularly concerning interventions and comparators in randomized trials.
- To guide systematic reviewers and guideline authors in evaluating evidence when trial conditions diverge from recommendation targets.
Main Methods:
- The Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group developed this concept article.
- An interdisciplinary panel utilized iterative review of examples, teleconferences, and group sessions.
- The paper was refined through expert feedback and approved following a presentation at a GRADE working group meeting.
Main Results:
- Differences between trial implementations and review targets in people, interventions, comparators, and outcomes are classified as indirectness.
- A specific subcategory of indirectness, 'indirectness of interventions and comparators,' arises when comparator groups receive the intervention (treatment switching).
- The proportion of participants switching and the magnitude of the effect influence the decision to rate down evidence quality due to indirectness.
Conclusions:
- Treatment switching and other discrepancies between intended and implemented interventions/comparators in trials are considered issues of indirectness.
- The GRADE framework provides a structured approach to evaluating and addressing indirectness in systematic reviews.
- Accurate assessment of indirectness is crucial for the validity and reliability of evidence-based recommendations.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:09Decision Making: P-value Method
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
01:24Testing a Claim about Population Proportion
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
01:50Statistical Significance
01:10P-value
P-value stands for the probability value. P-value is the probability that, if the null hypothesis is true, the results from another randomly selected sample will be as extreme or more extreme as the results obtained from the given sample.
A large P-value calculated from the data indicates to not reject the null hypothesis. But a higher P-value does not mean that the null hypothesis is true. The smaller the P-value, the more...
01:29Nursing Interventions II: Selecting and Classifying the Nursing Interventions