The equivalence of three data collection methods with field test data: a FACETS application

The present study demonstrated the utility of the FACETS software for evaluating items in the field test stage of item development for a clinical early childhood instrument. The research focus was the equivalence of the parent/caretaker interview, structured assessment, and observational methods for data collection for a developmental inventory for children from birth to age seven. Data for this study were from a field test with some missing responses. The Rasch-based software FACETS was used to test the equivalence of the methods of data collection as well as to identify items for which the methods did not provide equivalent information. Thirty-five items were studied from the adaptive domain, 26 items from the communication subdomain, 34 items from the motor domain, and 31 from the personal-social domain. When the methods were unconstrained, the overall test indicated that at least two of the methods were not equivalent. However, FACETS bias analyses with the method measures constrained to equality allowed the identification of a limited number of items and associated methods that were possibly problematic. The use of FACETS allowed test developers to focus on items from a field test event that were inconsistent with the targeted test development model.