Intervention effects in the case of heterogeneity between three subgroups. Assessment within the framework of...

Usually, statistical tests for interactions are applied to investigate potential effect modifiers. If an effect modifier, consisting of three categories, is found to be statistically significant, the application of pairwise interaction tests is indicated. In this case, the problem of non-transitive relations may occur if the significance level is fixed at 0.05 for all tests.

To develop an algorithm for which non-transitive relations do not occur.

A hierarchical testing procedure is applied, based on the heterogeneity statistic Q. In a first step the interaction will be tested for the three trial subgroups altogether, applying the significance level α = 0.05 (global test). If a significant interaction is proven in the first step, pairwise tests for interaction will be applied in a second step. Theoretical data scenarios will be considered and p-values will be calculated for the pairwise tests. Based on these results the significance level for pairwise testing will be determined.

Fixing the significance level at 0.05 for all tests, the problem of non-transitive relations is mostly relevant, if the difference in the effects between the three trial subgroups is approximately 3.5 standard errors and the effect of the 'middle' trial subgroup is not close to one of the other two effects. This problem vanishes when the significance level is set to α = 0.22. We propose to select α = 0.20 to get a more 'even' and simple value.

By increasing the significance level for the pairwise tests to 0.20, non-transitive relations are virtually avoidable. The proposed hierarchical testing procedure represents a clear practical guidance to perform subgroup analyses in the framework of systematic reviews.