The effect of dropping low scores on ability estimates

Dropping low scores is a common technique used in combining scores from multiple assessments, but no research has addressed the validity of ability estimates when dropping low scores. Using a simulation approach, the expected bias, root mean squared error (RMSE), and benefit to examinees resulting from dropping low scores was estimated for two ability estimation methods, Rasch estimation and proportion correct scoring. The simulation was done for three testing conditions: a normal condition; a bad day condition, when an examinee has a lower ability on one assessment; and a bad test condition, when one assessment is contaminated by an irrelevant factor. Ability estimates based on the complete data were generally preferable to estimates based on data with low scores dropped, suggesting that the use of dropping low scores is not warranted in most assessment situations.