Construct validity of scores/measures from a developmental assessment in mathematics using classical and many-facet...

Data from a developmental assessment comprised of 9 short answer mathematics tasks were validated using classical and three-faceted Rasch measurement methods. Field test data from a mixed age elementary school sample (N=280) were analyzed. Descriptive statistics on scores from the overall scale and two subdomains indicated improved performance with age. The data showed better fit with a two-factor model corresponding with the subdomain structure (Bentler's CFI=.94), than a one factor model (CFI=.87). The inter-factor correlation was.76. Convergent validity coefficients of scores with scaled scores of the Stanford AchievementTest mathematics battery ranged from.28 to.47; internal consistency reliability of the total and subdomain scores ranged from.87 to.89, respectively; and median inter-rater reliability was.75. On average, persons, tasks and raters showed acceptable fit with the three-facet Rasch model. Rasch logit difficulties of tasks suggested an ordered scale structure, although tasks tended to have high difficulty levels. The original and calibrated task ordering was consistent at the extreme ends of the scale. Gaps identified on the Rasch item map suggested a need for additional task construction. Conceptual and procedural differences in each technique are considered in deciding future improvements to the scale.