Model specification for nonlinearity and heterogeneity of regression in randomized pretest posttest studies:...

Area of Science:

• Psychology
• Statistics
• Quantitative Research Methods

Background:

• Randomized pretest posttest designs are prevalent in psychological research.
• Handling missing data in these designs, especially with violated statistical assumptions, remains a challenge.
• Existing methods often fall short when assumptions of statistical models are not met.

Purpose of the Study:

• To compare various analysis models for estimating average treatment effects in randomized pretest posttest designs with missing data.
• To investigate model performance under violations of linearity and homogeneity of regression slopes.
• To evaluate the impact of different missing at random (MAR) data patterns on these estimates.

Main Methods:

• Employed a randomized pretest posttest design framework.
• Compared listwise deletion and multiple imputation techniques for handling missing data.
• Assessed analysis models under violated assumptions of linearity and homogeneity of regression slopes.
• Examined several understudied MAR data patterns.

Main Results:

• Listwise deletion yielded unbiased and precise maximum likelihood estimates when the analysis model correctly handled violated assumptions and the pretest mean used all cases.
• Multiple imputation was effective provided the imputation model was correctly specified, underscoring the importance of model specification.
• The specific MAR data pattern significantly influenced results, highlighting the need to consider missingness patterns beyond the MAR assumption itself.

Conclusions:

• Effective handling of missing data in randomized pretest posttest designs, particularly with violated assumptions, requires careful consideration of both data handling techniques and analysis models.
• Listwise deletion can be a viable option if the analysis model is appropriately specified and pretest means are robustly estimated.
• Multiple imputation's success hinges on correct imputation model specification, and the nuances of MAR data patterns are critical for accurate average treatment effect estimation.