Evidence for the validity of a Rasch model technique for identifying differential item functioning

This paper presents an analysis of differential item functioning (DIF) in a certification examination for a medical specialty. The groups analyzed were (1) physicians from different subspecialties within this area and (2) physicians who qualified for the examination through two different experiential pathways. The DIF analyses were performed using a simple Rasch model procedure. The results were shown to be readily interpretable in terms of the known differences between the groups being compared. These results serve as validity evidence for the Rasch model procedure as a means for evaluating DIF in examinations. The conclusion is drawn that complex procedures are not required to generate interpretable results if relevant differences between the groups being compared are known. This suggests that the inability of many researchers to interpret results for racial/ethnic or gender groups is not due to inadequacies of the methods, but more likely to lack of pertinent knowledge about group differences.