Evaluating Equating Transformations in IRT Observed-Score and Kernel Equating Methods

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Test equating is crucial for score comparability across different test forms.
• Existing equating methods are based on Classical Test Theory (CTT) and Item Response Theory (IRT) frameworks.
• Comparing different equating methodologies is essential for understanding their performance and applicability.

Purpose of the Study:

• To compare the performance of three equating transformations: IRT Observed-Score Equating (IRTOSE), Kernel Equating (KE), and IRT Kernel Equating (IRTKE).
• To evaluate these methods under various data-generating scenarios, including a novel simulation procedure.
• To assess the impact of data properties like distribution skewness and item difficulty on equating accuracy.

Main Methods:

• Comparison of equating transformations derived from IRT Observed-Score Equating (IRTOSE), Kernel Equating (KE), and IRT Kernel Equating (IRTKE).
• Development of a new data-generation procedure for simulating test data without IRT parameters.
• Simulation studies to control for test score properties such as distribution skewness and item difficulty.

Main Results:

• Item Response Theory (IRT) methods generally yielded superior results compared to Kernel Equating (KE), even with non-IRT generated data.
• Kernel Equating (KE) showed potential for satisfactory results with appropriate pre-smoothing, offering significant speed advantages over IRT methods.
• The choice of equating method impacts results; sensitivity analyses are recommended for practical applications.

Conclusions:

• IRT-based equating methods are often more robust, particularly when data assumptions are not fully met.
• Kernel Equating (KE) can be a viable, faster alternative if pre-smoothing techniques are effectively implemented.
• For practical test equating, it is vital to consider model fit, assumption adherence, and the sensitivity of results to the chosen methodology.