Bias-Variance Trade-Off in Continuous Test Norming.

Area of Science:

• Psychometrics
• Statistical modeling

Background:

• Continuous test norming estimates score distributions using predictors.
• Generalized additive models (GAMs) offer a flexible approach for norm estimation.
• The impact of model flexibility and sample size on GAM estimates is not well understood.

Purpose of the Study:

• To investigate the relationship between model flexibility and bias, variance, and total variability in normalized z-score estimates.
• To evaluate these relationships under different population distributions (skewed t and normal) and assumption violations (transversal and longitudinal).

Main Methods:

• Utilized generalized additive models for location, scale, and shape (GAMLSS) for norm estimation.
• Simulated data from skew Student t and normal distributions.
• Assessed performance under transversal and longitudinal assumption violations.

Main Results:

• Overly strict distributional assumptions in models led to biased z-score estimates.
• Excessively flexible models resulted in increased variance of z-score estimates.
• The skew Student t distribution proved challenging to estimate for normally distributed data using the Box-Cox Power Exponential distribution.

Conclusions:

• Balancing model flexibility is crucial for accurate continuous test norming.
• Empirical norming practices should carefully consider model assumptions and flexibility to minimize bias and variance.
• Specific distributional assumptions, like the skew Student t, may require careful handling in norming procedures.