COMLEX-USA and in-service examination scores: tools for evaluating medical knowledge among residents

Between 2004 and 2006, the American Osteopathic Association added the evaluation of seven core competencies, including "Medical Knowledge," to the accreditation requirements for residency programs.

To determine whether scores on the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) and in-service examinations are useful in assessing osteopathic residents' medical knowledge.

Scores were gathered from medical education records from 2002 through 2006 for all residents in emergency medicine, family medicine, general surgery, internal medicine, and obstetrics and gynecology at Metro Health Hospital in Grand Rapids, Mich. Residents were assigned to one of four categories based on their score patterns on the three levels of COMLEX-USA. Score categories were compared with results from residents' in-service examinations. For subgroup analyses, statistical significance was assessed at P<.008.

A total of 74 residents took 170 in-service examinations during the study period. Among residents in the highest COMLEX-USA score pattern category, 70.7% of in-service examination scores were at or above the national mean. Only 20% of scores among residents in the lowest category were at or above the national mean. Of statistical significance, residents in the two highest COMLEX-USA categories had more test scores above the national mean than residents in the lowest category (P<.008). In addition, residents who took multiple in-service examinations showed an upward progression in scores, indicating an increase in residents' medical knowledge.

Residents' score patterns on COMLEX-USA generally correlated with their scores on in-service examinations, indicating that these examinations are useful assessment tools for measuring the "Medical Knowledge" core competency.