A comparison between robust z and 0.3-Logit Difference procedures in assessing stability of linking items for the...

Area of Science:

• Educational Measurement
• Psychometrics
• Statistical Modeling

Background:

• Assessing item difficulty stability is crucial for longitudinal assessments.
• The Rasch model is widely used in educational and psychological measurement.
• Common-item linking designs require stable items to maintain score comparability over time.

Purpose of the Study:

• To compare the performance of two procedures for assessing item difficulty stability within the Rasch model.
• To evaluate the agreement between the robust z procedure and the '.3 Logit Difference' procedure.
• To determine the suitability of these methods for identifying stable items in large-scale assessments.

Main Methods:

• Application of the robust z procedure and the '.3 Logit Difference' procedure to archival data.
• Analysis of item difficulty stability using data from the HSEE 1986/1987 and PACT 2004/2005 assessment programs.
• Calculation of the percentage of stable items identified by each procedure and the agreement between them.

Main Results:

• The '.3 Logit Difference' procedure identified slightly more stable items (2.6% more) than the robust z procedure.
• Approximately 93% of items were identically classified as stable or unstable by both methods, indicating high agreement.
• Both procedures demonstrated effectiveness in identifying stable items for common-item linking.

Conclusions:

• The high level of agreement suggests that either the robust z procedure or the '.3 Logit Difference' procedure can be reliably used for identifying stable items.
• The robust z procedure offers advantages due to its foundation in robust statistical inference and control of Type I error.
• The findings support the use of either method in common-item linking designs for maintaining score comparability in large-scale assessments.