Handling missing data in randomization tests for single-case experiments: A simulation study.

Area of Science:

• Psychology
• Educational Research
• Quantitative Methods

Background:

• Single-case experiments are prevalent in psychological and educational research.
• Missing or incomplete data frequently complicates the analysis of single-case experimental designs.
• Inadequate data handling can compromise the standards and validity of research findings, such as those set by the What Works Clearinghouse.

Purpose of the Study:

• To compare the effectiveness of different strategies for handling missing data in single-case experiments.
• To evaluate the impact of missing data on the type I error rate and statistical power of randomization tests.
• To identify optimal methods for data imputation in the context of various single-case designs.

Main Methods:

• Simulated complete datasets for ABAB phase, randomized block, and multiple-baseline designs.
• Introduced varying levels of missing data (10%, 30%, 50%) by random deletion.
• Evaluated three data handling strategies: randomized-marker method, single imputation (time series model), and multiple imputation (regression-based).

Main Results:

• The randomized-marker method demonstrated superior statistical power in randomization tests compared to single and multiple imputation.
• All evaluated methods successfully controlled the type I error rate under simulated conditions.
• The performance of imputation strategies varied across different levels of missing data.

Conclusions:

• The randomized-marker method is the recommended strategy for handling missing data in single-case experiments when statistical power is a primary concern.
• This method ensures that randomization tests maintain acceptable type I error rates while maximizing power.
• Researchers should carefully consider data handling strategies to uphold the methodological rigor of single-case research.