An Investigation of the Properties of Computer-Based Case Simulations

Computer-based case simulations (CCS) require several development iterations, each involving a sizeable investment of time from physician volunteers and test development staff. Hence, a case that fails to demonstrate good measurement properties when administered to examinees at the pretest stage represents a costly loss. An earlier study investigated the relationship of case properties to case difficulty in part to obtain information that would permit (a) better instructions to case developers and (b) identification of problematic cases earlier in the developmental cycle. Some of the best predictors, however, were scoring points typically obtained at the pretest stage, making them inappropriate for meeting these two goals. The objective of this study was to determine if these scoring points could be predicted from cases properties that would be available early in case development. Case description variables and scoring points were available for 28 cases which were analyzed using regression procedures. Three models were identified that predicted the important scoring fairly well. These models differed primarily in the area of medicine: internal medicine, obstetrics/gynecology, or no medical area specified. The results were consistent with those found for predicting difficulty and appeared promising for better understanding of the nature of the simulations.