Longitudinal stability of IRT and equivalent-groups linear and equipercentile equating

Area of Science:

• Educational Measurement
• Psychometrics
• Developmental Psychology

Background:

• Accurate measurement of learning requires equivalent test forms.
• Equating methods are used to ensure scores from different test forms are comparable.
• Head Start programs serve young children, necessitating reliable assessment tools.

Purpose of the Study:

• To compare the effectiveness of three equating methods: common-item IRT equating, linear transformation, and equipercentile transformation.
• To evaluate these methods over an academic year using data from 1,667 Head Start children.
• To identify which equating method minimizes discrepancies between two presumably equivalent test forms (A and B).

Main Methods:

• A multiscale criterion-referenced test with two forms (A and B) was administered to 1,667 children over four points in an academic year.
• A randomly equivalent groups design was employed.
• Common-item IRT equating (concurrent calibration), linear transformation, and equipercentile transformation were used for equating.

Main Results:

• Item Response Theory (IRT) equating demonstrated different discrepancy patterns compared to linear and equipercentile methods over time.
• IRT equating resulted in marginally smaller mean score differences between forms A and B.
• IRT equating generated slightly fewer distributional discrepancies between the two test forms than linear and equipercentile equating.

Conclusions:

• IRT equating appears to offer a slight advantage in minimizing form-to-form score discrepancies and distributional differences.
• The results suggest mixed findings, indicating that no single method is universally superior.
• Additional research is necessary to fully elucidate the relative strengths and weaknesses of IRT, linear, and equipercentile equating methods in practice.