Does Instruction Affect the Underlying Dimensionality of a Kinesiology Test?

Area of Science:

• Educational Measurement
• Kinesiology
• Psychometrics

Background:

• Effective instruction aims to enhance student knowledge and cognitive functions.
• The impact of instructional changes on the dimensionality of achievement tests remains underexplored.
• Understanding test dimensionality is crucial for accurate measurement of student achievement.

Purpose of the Study:

• To investigate whether effective instruction influences the dimensionality of a kinesiology achievement test.
• To examine the stability of item difficulties and potential threats to measurement equivalence.
• To explore the relationship between item content, test position, and dimensionality.

Main Methods:

• A quasi-experimental pre- and post-test design with 108 college students.
• Rasch dichotomous model for item parameterization (n=42 items).
• Principal Component Analysis (PCA)/t-test protocol for dimensionality threats and statistical decomposition of item residuals.

Main Results:

• Significant student achievement gains were observed between pre- and post-testing.
• Dimensionality disturbances were minor, with less than ten percent of items showing significant shifts.
• An unexpected residual cluster of technical content items near the test's end suggested test position as a measurement equivalence threat.

Conclusions:

• Effective instruction had a relatively minor impact on the dimensionality of the kinesiology achievement test.
• Conventional measurement equivalence methods may overlook multiple, varied sources of variability.
• Further research is needed to address complex, multi-faceted threats to measurement equivalence in educational assessments.