Translation equivalence across PISA countries

Due to the continuous increase in the number of countries participating in international comparative assessments such as TIMSS and PISA, ensuring linguistic and cultural equivalence across the various national versions of the assessment instruments has become an increasingly crucial challenge. For example, 58 countries participated in the PISA 2006 Main Study. Within each country, the assessment instruments had to be adapted into each language of instruction used in the sampled schools. All national versions in languages used for 5 per cent or more of the target population (that is, a total of 77 versions in 42 different languages) were verified for equivalence against the English and French source versions developed by the PISA consortium. Information gathered both through the verification process and through empirical analyses of the data are used in order to adjudicate whether the level of linguistic equivalence reached an acceptable standard in each participating country. The paper briefly describes the procedures typically used in PISA to ensure high levels of translation/adaptation accuracy, and then focuses on the development of the set of indicators that are used as criteria in the equivalence adjudication exercise. Empirical data from the PISA 2005 Field Trial are used to illustrate both the analyses and the major conclusions reached.