A deterministic theory of clinical performance rating: promising early results

The proposed theory provides a basis for both measuring and correcting rater stringency error in some grossly incomplete rating data matrices. The theoretical model fit (R = .92, .85, .82; joint p less than .000001) ratings made by faculty and resident physicians (n = 47, 31, and 29) of student clinical performance in each of three junior year medical student cohorts (n = 29, 30, and 35) better than alternative models. In these data the percentage of variation attributable to stringency and ability was about 35 and 40, respectively. Three-month test-retest reliability for rater stringencies was .16 less than r less than .29 (joint p less than .04). Cross-validation supported the proposed model (r = .61) over the conventional alternative (r = .41; z = 2.62, p less than .004). Both reliability and convergent validity of the ability construct were .20 greater for one corrected rating than for one observed (uncorrected) rating.