Understanding heterogeneity of pragmatic trial results and subgroup analyses: The FIAT example

Pragmatic trial results are intrinsically heterogeneous and the 'average treatment effect', on which the paradigmatic verdict of explanatory trials is based may not suffice to translate trial results into clinically meaningful conclusions applicable in practice. Examining various subgroups is problematic because they are at risk of both false negative and false positive results.

We summarize FIAT, a pragmatic care trial on flow diversion where multiple subgroups were examined. The notions of average treatment effect and interaction tests are reviewed to better understand their application in pragmatic trials.

The trial showed flow diversion to be superior to standard treatments, but the results do not apply to all intracranial aneurysms. The notion of a 'true average treatment effect' can hardly apply when there are multiple comparator interventions and clinical heterogeneity. Various subgroups were examined in spite of negative interaction tests, to help interpret the trial results. Subgroup findings can be credible so long as they are not data-dependent 'fishing expeditions'. Meaningful clinical subgroups that have been pre-specified and integrated into the randomization scheme and power calculation provide the most credible conclusions.

Non-prespecified data-dependent subgroup analyses are at high risk of being incorrect and should not be used to make clinical decisions in practice. A critical assessment of pre-specified subgroup analyses can nonetheless help interpret heterogeneous pragmatic trial results.