Angoff anchor statements: setting a flawed gold standard?

Area of Science:

• Educational Measurement
• Psychometrics
• Standard Setting

Background:

• The Angoff method is a widely used standard-setting procedure.
• Anchor statements are crucial for the Angoff method.
• Current interpretations of anchor statements vary in practice.

Purpose of the Study:

• To critically review the conceptualization and interpretation of anchor statements in the Angoff standard setting method.
• To propose a refined approach for utilizing anchor statements in determining performance standards.

Main Methods:

• Conceptual analysis of anchor statement interpretation in standard setting.
• Review of existing practices in applying the Angoff method.
• Argument for a revised focus in anchor statement design.

Main Results:

• Current practices often overemphasize the perceived difficulty of anchor statements for specific candidate subgroups.
• A more appropriate approach involves focusing on the importance of achieving a standard and ensuring it applies to all candidates.
• Refined anchor statements should estimate 'minimum acceptable performance by every candidate' for each test item.

Conclusions:

• The Angoff standard setting method can be improved by refining the use of anchor statements.
• Shifting focus from test difficulty to the importance of universal minimum performance is recommended.
• Implementing revised anchor statements will lead to more consistent and valid cut-score derivations.