A Simulation Study on the Performance of the Simple Difference and Covariance-Adjusted Scores in Randomized...

Area of Science:

• Statistics
• Experimental Design
• Psychometrics

Background:

• Huck and McLean (1975) established the covariance-adjusted score's superiority over the simple difference score.
• Current research indicates inconsistent application of these scores in two-wave randomized designs.

Purpose of the Study:

• To investigate the power of simple difference and covariance-adjusted scores under relaxed assumptions.
• To identify conditions favoring the use of either score type in detecting treatment effects.

Main Methods:

• A Monte Carlo simulation was employed.
• Manipulated factors included sample size, distribution normality, pretest-posttest correlation, and posttest variance.
• 226,000 unique samples were generated across 1,000 replications per condition.

Main Results:

• The gain score demonstrated power comparable to the covariance-adjusted score when pretest and posttest variances were equal.
• The gain score was equally powerful in fan-spread growth conditions.
• The relative power varied based on the manipulated factors.

Conclusions:

• The simple difference (gain) score is a viable alternative to the covariance-adjusted score in certain two-wave randomized experimental designs.
• Equal variances and fan-spread growth conditions support the use of the gain score.
• Researchers should consider these conditions when selecting a score type.