Common reporting errors in subgroup analysis: a comparison of interaction and stratified regression models.

Area of Science:

• Epidemiology
• Public Health
• Medical Research

Background:

• Reporting treatment effect estimates from stratified regressions alongside significance test results from interaction regressions is common practice.
• However, combining these methods raises methodological concerns due to differing theoretical foundations and statistical properties.
• This study evaluates their effectiveness in estimating heterogeneous treatment effects.

Purpose of the Study:

• To compare the performance of interaction regression and stratified regression in estimating heterogeneous treatment effects.
• To identify scenarios where each method excels or struggles.
• To provide guidance on appropriate reporting strategies for subgroup analyses.

Main Methods:

• Conducted Monte Carlo simulations varying sample size, subgroup proportions, covariate-outcome heterogeneity, and covariate correlation.
• Evaluated performance using mean squared error (MSE), empirical coverage rate (ECR), and empirical statistical power (ESP).
• Applied both methods to real-world data from the International Social Survey Programme (ISSP) for illustrative purposes.

Main Results:

• Stratified regression effectively controlled Type I error with large sample sizes and balanced groups.
• Interaction regression struggled with Type I error control when baseline characteristics differed and covariates were weakly correlated.
• Interaction regression generally outperformed stratified regression in other scenarios, offering higher statistical power with acceptable Type I error rates.

Conclusions:

• Interaction regression and stratified regression have distinct strengths and limitations for estimating heterogeneous treatment effects.
• Researchers should avoid hybrid reporting strategies combining estimates from one method with significance tests from another.
• Differences in Type I error control underscore the need for careful method selection and reporting in subgroup analyses.