An ecological view of measurement: focus on multilevel model explanation of differential item functioning

The 3rd Regional Comparative and Explanatory Study reports, analyses and compares academic results in mathematics, sciences, and reading for 15 Latin American countries. Validity is the foundation of a testing procedure, and the process of validation is important to the overall success of educational assessment as a whole. This methodological study deals specifically with an ecological point of view which includes and situates the person, process, context, and time of the testing situation. These descriptions indicate specific events showing how and what variables at the individual, school, or country level can give a deep understanding of the response process. The aims were to investigate ecological explanations of differential item functioning.

The study analysed the science test given in 2013 to 6th grade students and the data pool consisted of 12,657 students from 2,609 schools and 15 countries. A progressive inclusion of the variance distribution in different Bernoulli logistic regression models was carried out.

The analyses showed the presence of differential item functioning in 32% of the science items.

The main source of differential item functioning was related to the human development level of the participating countries.