Maintaining instrument quality while reducing items: application of Rasch analysis to a self-report of visual...

While efficiency has been of concern in the measurement of health care outcomes, little attention has been devoted to methods that achieve efficient, shortened instruments that have good psychometric properties. The purpose of this study was to show how Rasch analysis could be used to reduce the number of items in an instrument while maintaining credible psychometric properties. This approach was applied to the Visual Function-14 (VF-14), a self-report of 14 vision-dependent activities, designed to measure the need for and outcomes of cataract surgery. An instrument which contained the VF-14 plus an additional 10 items that were developed for the study (VF-24) was administered to sixty-one patients (73.7+/-9.5 years) about to undergo extracapsular cataract removal at one of two surgical centers. Rasch analysis (BIGSTEPS) of the VF-14 showed a number of limitations to the original instrument, including: 1) unequal use of the five rating categories, 2) ceiling effect, 3) several other gaps where patient abilities did not match with item difficulties, and 4) sets of items that appeared redundant, (i.e., having the same calibration level). To resolve the first three of these problems, the rating scale was converted to a three-point scale and BIGSTEPS was run with all 24 items. (10 additional items added to the VF-14 designed to "fill in" the gaps). The conversion to a three-point scale and the increase in items resulted in some improvement in the matching of item difficulty to patient ability, as evidenced by a slight decrease in gaps. The addition of items resulted in improvements in person separation (2.55 to 2.99) and Cronbach's alpha (.83 to .91) but did not substantially reduce the ceiling effect and furthermore resulted in an increase in item redundancy. The final practical improvement undertaken was to reduce the number of items while attempting to maintain the psychometric qualities of the instrument as a whole. Three criteria were used in deciding to remove items: 1) high mean square, 2) low mean square and 3) items having similar calibrations. In addition, if an analysis showed that the removal of an item substantially decrease person separation, that item was retained for further analyses. Relative to the original VF-14, the resulting VF-10 showed less redundancy of items while person separation (2.20) and Cronbach's alpha (.89) remained relatively intact. The study demonstrates that Rasch analysis, while effective in elucidating the metrics of an original instrument, can also be useful in designing modifications of instruments that are both efficient and psychometrically sound.