An analysis of the reliability of a laboratory evaluation system

Many investigators have reported attempts to develop reliable laboratory and clinic evaluation systems. However, few studies, regardless of level of success, have used an analytic procedure to identify those components of the evaluation system that, if refined further, could improve reliability. This paper described the development of an evaluation system for two preclinical laboratory courses. Reliability estimates of 0.80, 0.74, and 0.68 were obtained for three different evaluation instruments used in conjunction with the evaluation system. An analytic technique for identifying components contributing to, and detracting from, the reliability of the evaluation system is also described. Further refinements based on this analysis are discussed.