The ISR: intelligent student reports

Rasch-based Scale Scores are a simple linear transformation of the basic logit metric. Scale Scores are the quantification of the measurement continuum. This quantification makes it possible to do arithmetic, computer differences, and apply standard statistical techniques. However, qualitative meaning is not in the numbers and must come from experience with the scale and from the descriptive information that can (and should) be attached. This includes item content and exemplars, normative information for relevant groups, historical data for the individual, and evaluative assessment like performance levels standards. The Scale Score metric is the structure that manages the organization of intelligent reports and recognizes anomalies. Scale Scores have no meaning, per se, but can provide a strong framework for organizing useful reports and presenting meaningful information. They facilitate diagnosis by "Analysis of Fit" and by "Analysis of Misfit." The Analysis of Fit relies on the general definition of the construct to describe what a student at a particular point on the scale can and cannot do. It is meaningful to the extent that the student conforms to the expectations of the measurement model. The Analysis of Misfit uses the model to identify surprises, i.e., departures from the model expectations. It highlights atypical areas of strong and weak performance. The intent is to bring these exceptons to the attention of the experts for informed, substantive interpretation and diagnosis. Intelligent reports, to be useful, and to justify the time and expense of testing, need to provide more information in a useable format than the candidate, student, parent, or educator had available otherwise. This requires more than reporting a single number or a single decision. It should include sufficient scaffolding to allow the consumer to extract quickly and efficiently all the useful information that can be taken from the test. Rasch Scale Scores are an important, perhaps essential, tool in this process.