Optimal Test Design for Estimation of Mean Ability Growth

Area of Science:

• Educational Measurement
• Psychometrics
• Statistics

Background:

• Accurate measurement of student ability growth is essential for educational assessment.
• Test design, specifically item difficulty allocation, significantly impacts measurement precision.
• Existing methods may lack robustness in estimating ability growth across different populations.

Purpose of the Study:

• To introduce an optimal test design methodology for estimating mean and percentile ability growth with high precision.
• To investigate the influence of item discrimination and ability growth magnitude on optimal item difficulty allocation.
• To evaluate the robustness of the proposed design against uncertainties in mean ability growth.

Main Methods:

• Utilizing the asymptotic expression of variance in relation to test information for optimization.
• Employing particle swarm optimization (PSO) to determine the optimal allocation of item difficulties.
• Implementing discrete optimization using simulated annealing for practical test assembly from item pools.

Main Results:

• Optimal item difficulty allocation is contingent upon item discrimination and the magnitude of ability growth.
• The optimization function's dependence on examinee abilities necessitates considering the unknown mean ability growth.
• An optimum in-average design demonstrates robustness against uncertainty in mean ability growth.

Conclusions:

• Particle swarm optimization provides an effective method for designing achievement tests to measure ability growth.
• The proposed test design method enhances the precision of estimating both mean and percentile ability growth.
• The study highlights the importance of considering item parameters and population characteristics for robust test design.