Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives

Area of Science:

• Biostatistics
• Clinical Trials Methodology
• Epidemiology

Background:

• Subgroup analyses are frequently conducted in randomized controlled trials (RCTs).
• Despite existing guidelines, inappropriate subgroup analyses persist in scientific literature.
• Misleading subgroup findings can lead to incorrect treatment decisions.

Purpose of the Study:

• To evaluate the potential for misleading results from subgroup analyses.
• To compare subgroup-specific analyses with formal tests of interaction.
• To determine factors influencing subgroup analysis performance and recommend best practices.

Main Methods:

• Simulated data for RCTs with and without differential subgroup effects.
• Assessed subgroup-specific analyses and formal interaction tests for accuracy.
• Explored impact of sample size, effect magnitude, and data variability on analysis performance.

Main Results:

• Subgroup-specific tests were less reliable than formal interaction tests, often yielding false positives.
• Formal interaction tests demonstrated superior performance in correctly identifying differential effects.
• Detecting subtle subgroup effects requires substantial sample size increases, often exceeding the 'rule of four'.

Conclusions:

• Subgroup analyses, particularly subgroup-specific tests, are prone to spurious findings and their extent is underestimated.
• Formal tests of interaction are preferred but should be interpreted cautiously, especially for subtle effects.
• Recommendations include pre-specifying subgroup analyses, powering trials for them, and viewing results as hypothesis-generating.