The assessment of professional competence

Valid assessment of professional competence has proven to be an elusive goal. Objective tests, direct observation of performance, overall ratings of competence, and simulations have been tried and found wanting in one way or another. Objective test items are criticized as being unrealistic and therefore invalid. Direct observation tends to be very unreliable and therefore invalid. Simulations and overall ratings of competence share both of these flaws to some extent. Basically, you can't win. This article outlines some of the many ways to lose and some ways to cut these losses. In doing so, it proposes a general framework for evaluating the validity of measures of competence, and it uses this framework to examine the strengths and weaknesses of three approaches to the assessment of professional competence: direct observation, simulation, and objective testing.