On the factor structure of standardized educational achievement tests

Area of Science:

• Educational Measurement
• Psychometrics
• Educational Psychology

Background:

• Standardized educational achievement tests (SEAT) are crucial for high-stakes educational accountability assessments in California.
• Understanding the factor structure of these tests is essential for accurate score interpretation and psychometric analysis.

Purpose of the Study:

• To analyze the item- and subtest-level factor structure of California's SEATs.
• To investigate the dimensionality and variance accounted for at different analysis levels.
• To examine the implications for test calibration, scaling, and score interpretation.

Main Methods:

• Full information factor analysis.
• Multidimensional item response theory (IRT) models, including TESTFACT and NOHARM.
• Analysis of item- and subtest-level data from California's SEATs.

Main Results:

• SEATs demonstrate high unidimensionality at the item level, accounting for only 1/4 to 1/3 of the variance.
• Subtest-level analysis reveals a richer factor structure, explaining approximately 2/3 of the total common variance.
• Factors at both levels were highly correlated, indicating strong second-order and general factors.

Conclusions:

• While items may appear unidimensional, subtest analysis provides a more comprehensive view of test structure.
• The presence of strong general factors suggests a common underlying ability influencing performance across diverse content areas.
• Findings have significant implications for the assumptions and application of modern psychometric models in educational assessment.