Comparing the Weighted Gain Score and a Rasch-Based Approach for Estimating Learning Outcomes in Medical Education:...

Area of Science:

• Medical Education Research
• Educational Measurement
• Statistical Analysis

Background:

• Pretest-posttest designs are common for evaluating educational interventions in medical settings.
• The Weighted Gain Score (WGS) was developed to mitigate bias from baseline performance differences.
• Estimating learning gain accurately is crucial for assessing educational effectiveness.

Purpose of the Study:

• To evaluate the statistical and inferential properties of the Weighted Gain Score (WGS).
• To compare the WGS with Rasch Learning Gain (RLG) across diverse datasets.
• To assess the impact of the WGS parameter µ on learning gain calculations.

Main Methods:

• The WGS was analyzed using three datasets: illustrative, empirical, and simulated.
• The effect of varying the WGS parameter µ (30, 50, 70) on learning gain was examined.
• WGS results were compared against Rasch Learning Gain (RLG) for statistical properties.

Main Results:

• Varying the WGS parameter µ influenced the magnitude of learning gain estimates.
• Despite scale differences, WGS and RLG showed strong correlations in empirical and simulated data (r=0.93 and r=0.92, respectively).
• Changes in µ did not affect inferential outcomes; both methods identified the same interaction effect.

Conclusions:

• The WGS provides results consistent with RLG but with significantly lower computational complexity.
• The WGS is applicable to both small and large datasets.
• The parameter µ allows for calibration of learning gain estimates across cohorts without altering inferential conclusions.