Translation equivalence across PISA countries

Area of Science:

• Educational Measurement
• International Comparative Studies
• Psychometrics

Background:

• Increasing global participation in international assessments (e.g., PISA) necessitates robust methods for ensuring assessment quality across diverse linguistic and cultural contexts.
• The adaptation of assessment instruments into multiple languages presents significant challenges for maintaining conceptual and psychometric equivalence.
• Verification of national versions against source documents is critical for data integrity in large-scale international studies.

Purpose of the Study:

• To describe the procedures used in the Programme for International Student Assessment (PISA) to ensure high levels of translation and adaptation accuracy.
• To focus on the development of indicators used as criteria for adjudicating the linguistic equivalence of national assessment versions.
• To evaluate the effectiveness of established procedures using empirical data from the PISA 2005 Field Trial.

Main Methods:

• Development and application of a set of indicators to assess linguistic equivalence.
• Verification of national assessment versions against English and French source versions.
• Empirical analyses of data from the PISA 2005 Field Trial to illustrate adjudication processes.

Main Results:

• The study outlines the criteria and indicators employed in PISA for evaluating the equivalence of translated assessment instruments.
• Empirical data analysis demonstrates the application of these indicators in the adjudication process.
• Findings inform the decision-making regarding the acceptability of linguistic equivalence for national versions.

Conclusions:

• The described procedures and indicators are crucial for maintaining the quality and comparability of international assessment data.
• Successful linguistic and cultural equivalence is achievable through rigorous verification and empirical analysis.
• The study contributes to best practices in developing and adapting large-scale international educational assessments.