Related Experiment Videos
Some links between classical and modern test theory via the two-level hierarchical generalized linear model
1Department of Educational Leadership and Policy Studies, School of Education, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA. yasuom@vt.edu
Journal of Applied Measurement
|June 9, 2005
Summary
Classical Test Theory (CTT) and Modern Test Theory (MTT) can be unified using a hierarchical generalized linear model (HGLM). This framework reveals differences in sampling models and link functions, impacting specific objectivity in measurement.
Area of Science:
- Psychometrics
- Educational Measurement
- Statistical Modeling
Background:
- Classical Test Theory (CTT) and Modern Test Theory (MTT), including Item Response Theory (IRT) and Rasch models, offer different perspectives on test analysis.
- Reconciling these theories is crucial for a comprehensive understanding of measurement properties.
- Hierarchical Generalized Linear Models (HGLMs) provide a flexible framework for analyzing nested data structures.
Purpose of the Study:
- To explore the connections between Classical Test Theory (CTT) and Modern Test Theory (MTT) within the HGLM framework.
- To reformulate CTT and MTT models as HGLMs, treating items nested within subjects.
- To contrast the Rasch and two-parameter IRT models concerning specific objectivity.
Main Methods:
- Reformulation of CTT and MTT models as two-level HGLMs, with item difficulty as fixed effects and subject ability as random effects.
- Analysis of differences in level 1 sampling models and link functions within the HGLM.
- Reanalysis of English composition scores using the HGLM framework to illustrate theoretical concepts.
Main Results:
- Both CTT and MTT models can be represented as HGLMs, differing primarily in their level 1 sampling model and link function.
- The Rasch model and the two-parameter IRT model exhibit distinct properties regarding specific objectivity.
- Essentially tau-equivalent models demonstrate specific objectivity when data fit the model, unlike congeneric measures models.
Conclusions:
- The HGLM framework offers a unified approach to understanding CTT and MTT, highlighting their underlying structural similarities and differences.
- Specific objectivity is a key differentiator between IRT models, contingent on model-data fit and the type of measurement model used.
- This integrated perspective enhances the theoretical and practical application of psychometric models in educational and psychological assessment.