Examining classification criteria: a comparison of three cut score methods

Area of Science:

• Psychometrics
• Educational Psychology
• Child and Adolescent Psychiatry

Background:

• Accurate screening for behavioral and emotional disorders in children is crucial for timely intervention.
• Establishing reliable cut scores for screening instruments is essential for valid classification.
• The Behavioral and Emotional Screening System (BESS) Teacher Rating Scale is a widely used tool for child and adolescent behavioral assessment.

Purpose of the Study:

• To compare the effectiveness of three distinct methods for establishing cut scores on the BESS Teacher Rating Scale.
• To evaluate T scores, Receiver Operating Characteristic (ROC) curve analysis, and the Rasch Rating Scale Method (RSM) for optimal cut score determination.
• To assess the concordance and accuracy of these methods using norm data and clinical diagnostic information.

Main Methods:

• Utilized the normative dataset of the BESS Teacher Rating Scale for Children and Adolescents.
• Compared T scores, ROC analysis, and RSM across seven classification indices.
• Validated cut score accuracy using a subset of children with pre-existing disorder diagnoses.

Main Results:

• All three methods demonstrated general concordance in identifying cut scores.
• RSM and ROC analysis yielded highly similar results, pinpointing the same optimal cut-point.
• The T score method produced a more conservative optimal cut score, identifying a lower threshold.

Conclusions:

• RSM and ROC analysis offer robust and comparable approaches for setting cut scores on the BESS Teacher Rating Scale.
• The T score method provides a more conservative alternative, potentially reducing false positives.
• The choice of method may depend on the specific goals of the screening and the desired balance between sensitivity and specificity.