Estimating True Student Growth Percentile Distributions Using Latent Regression Multidimensional IRT Models.

Area of Science:

• Educational Measurement
• Psychometrics
• Latent Variable Modeling

Background:

• Student Growth Percentiles (SGPs) are increasingly used for student achievement growth and educator effectiveness evaluations in the U.S.
• Emerging research indicates significant measurement errors in SGPs estimated from observed test scores.
• Limited understanding exists regarding the properties of "true" SGPs, defined by latent achievement attributes.

Purpose of the Study:

• To develop a novel framework for studying the distributional properties of true SGPs.
• To provide new empirical information about true SGPs using longitudinal data.
• To evaluate the accuracy of SGPs and their relationships with student and teacher characteristics.

Main Methods:

• Developed a novel framework using latent regression multidimensional item response theory (IRT) models.
• Applied the framework to longitudinal item response data from over 330,000 students.
• Analyzed distributional properties of true SGPs and their correlations with student characteristics.

Main Results:

• True SGPs showed correlations of 0.3 to 0.5 across mathematics and English language arts.
• True SGPs exhibited nontrivial relationships with student race/ethnicity and absenteeism.
• Accuracy gains from incorporating these relationships into observed SGPs were modest.
• Average true SGPs for teachers varied substantially based on student demographics.

Conclusions:

• The study provides crucial insights into the nature and limitations of Student Growth Percentiles.
• Findings highlight the need for careful interpretation of SGPs, especially in educator evaluations.
• The developed IRT framework offers a robust method for analyzing true SGPs and their influencing factors.