Validating standard setting with a modified nedelsky procedure through common item test equating

Area of Science:

• Medical Education
• Psychometrics
• Educational Measurement

Background:

• Validating cut scores for high-stakes tests is challenging due to limited common items between forms.
• Common item equating offers a method to assess score stability over time with a stable item pool.

Purpose of the Study:

• To evaluate the stability of cut scores determined by the modified Nedelsky procedure.
• To compare judged cut scores with equated cut scores derived from Rasch common item equating.

Main Methods:

• Analysis of five forms of a basic science examination over three years.
• Application of the Rasch common item equating technique.
• Comparison of modified Nedelsky procedure cut scores with Rasch-equated cut scores.

Main Results:

• The modified Nedelsky procedure cut scores demonstrated stability across five administrations.
• Judged cut scores were consistently within the equating error of Rasch-equated scores.
• The Rasch common item equating technique provided a reliable method for stability analysis.

Conclusions:

• Cut scores set using the modified Nedelsky procedure are stable and reliable for this basic science examination.
• Common item equating using the Rasch model is a viable method for validating judged cut scores in large-scale assessments.
• This methodology supports the consistency of performance standards in high-stakes testing programs.