Value-added scores show limited stability over time in primary school

Area of Science:

• Educational measurement
• Educational accountability
• Quantitative psychology

Background:

• Value-added (VA) models are crucial for assessing educational effectiveness and accountability.
• Concerns exist regarding the stability and domain-specificity of VA scores, potentially leading to unfair high-stakes decisions.
• Limited research has jointly investigated school-level VA score stability over time and across different academic domains.

Purpose of the Study:

• To examine the longitudinal stability of school-level value-added scores in mathematics and language learning.
• To assess the consistency of school VA scores across different academic domains.
• To evaluate the suitability of VA scores for high-stakes decision-making in education.

Main Methods:

• Utilized representative, large-scale, longitudinal data from two cohorts of standardized achievement tests in Luxembourg.
• Analyzed data from 7,016 students across 151 schools.
• Calculated rank correlations of VA scores from 2017 to 2019 for mathematics and language learning.

Main Results:

• Only 34-38% of schools demonstrated stable VA scores over the study period.
• Moderate rank correlations were observed: r = .34 for mathematics and r = .37 for language learning.
• VA scores exhibited insufficient stability for high-stakes accountability decisions.

Conclusions:

• School-level VA scores lack the necessary temporal stability for high-stakes accountability.
• Despite limitations, VA scores can be valuable for identifying effective teaching and school practices, particularly in diverse student populations.
• Further research is needed to refine VA models for reliable educational assessment.