The Evidence for a Subscore Structure in a Test of English Language Competency for English Language Learners

Area of Science:

• Psychometrics
• Educational Measurement
• Psychological Testing

Background:

• Reporting subscores for tests designed for unidimensional scales is a growing area of interest.
• Existing methods may not fully capture nuanced score structures.
• Multidimensional item response theory (MIRT) offers advanced modeling capabilities.

Purpose of the Study:

• To apply MIRT to identify a subscore structure within a unidimensional test.
• To address challenges of planned missing data in multi-form testing.
• To demonstrate the generalizability and utility of identified subscores.

Main Methods:

• Application of multidimensional item response theory (MIRT).
• Analysis of planned missing data designs with low item overlap.
• Validation of multidimensional structure across multiple test forms.
• Comparative analysis of subscores across different examinee groups.

Main Results:

• Successfully identified a meaningful subscore structure within a unidimensional test.
• Demonstrated the generalizability of this structure across multiple test forms.
• Showcased the utility of subscores in differentiating examinee groups.
• Confirmed that subscores provide valuable information on distinct constructs, even when the overall data fit a unidimensional model.

Conclusions:

• Subscores can be effectively computed and reported from tests designed for unidimensional reporting.
• MIRT provides a robust framework for uncovering latent multidimensionality.
• Identified subscores offer enhanced interpretability and utility for understanding examinee characteristics across diverse groups.