Development of the knowledge-based standard for the written certification examination of the American Board of...

In 1988 and 1989, the American Board of Anesthesiology (ABA) developed a knowledge-based standard for its written certification examination. In brief, 13 "judges" developed a construct of a "borderline candidate," i.e., a candidate who was neither ideal nor clearly failing but rather had sufficient knowledge to just pass. In 1989, this construct was applied to 90 questions from the 1989 ABA examination to estimate candidate's score on that subset. When extended to the entire examination, the use of the construct resulted in a knowledge-based standard of 57% correct. (The 1988 exercise, also using the construct of a borderline candidate but with a totally different subset of questions, produced an identical standard). This standard resulted in higher success rates among the actual examinees taking the ABA examination (84% in 1989 and 90% in 1990) than had the normative standard used previously (80%). The authors suggest that the process they describe permits development of a reproducible criterion for success that is based entirely on mastery of a relevant body of knowledge rather than on normative considerations.