Content validation of key features on a national examination of clinical decision-making skills

Key features (KFs) represent the critical, or essential, steps in the identification and management of a clinical problem. KFs for 59 clinical problems were defined by members of a test committee for the Medical Council of Canada as part of their efforts to create a more valid written examination of clinical decision-making skills for the Canadian Qualifying Examination in Medicine. In order to evaluate the content validity of KFs that the test committee had defined for the examination, 99 physicians from outside the committee, who came from clerkship programs at all 16 of Canada's medical schools, participated in three studies conducted in 1991. The first study was retrospective and was designed to find the degree of agreement or disagreement that the outside physicians had with the KFs already defined for each problem by the committee members. The second study was prospective and was to compare the KFs generated de novo by the participants with those already defined by the committee members. The third study was to gather the outside physicians' opinions of the frequencies with which graduating students in Canada are exposed to the 59 problems used in the retrospective and prospective studies. Almost all the KFs defined by the test committee were corroborated by the outside physicians, 92% in the retrospective study and 94% in the prospective one.(ABSTRACT TRUNCATED AT 250 WORDS)