A Validity Study of COMLEX-USA Level 2-CE and COMAT Clinical Subjects: Concurrent and Predictive Evidence

The Comprehensive Osteopathic Medical Licensure Examination (COMLEX-USA) Level 2-Cognitive Examination (CE) and the Comprehensive Osteopathic Medical Achievement Test (COMAT) are administered to similar populations (third- and fourth-year osteopathic students) at similar points in time. Examining the relationship between scores on the 2 assessments that measure similar constructs ultimately supports the validity of both.

The purpose of this study is to provide empirical evidence of the concurrent and predictive validity of COMAT and COMLEX-USA Level 2-CE.

In 2018, first-attempt scores on Level 2-CE were aggregated from June 2015 to May 2018 and matched with first-attempt scores on each COMAT clinical subject. We conducted correlational analyses between performance on COMAT and Level 2-CE, and COMAT scores and Level 2-CE discipline subscores. Additionally, we used multivariate regression to analyze the predictive relationship between performance on all COMAT clinical subjects and Level 2-CE.

The results from correlational analyses indicated statistically significant, positive associations between COMAT and Level 2-CE scores (r = 0.49-0.68, P < .0001), and statistically significant, but slightly weaker relationships between COMAT scores and Level 2-CE discipline subscores (r = 0.31-0.60, P < .0001). Furthermore, results from the multiple regression indicated that scores on COMAT explained 68% of the variance in Level 2-CE scores, and that COMAT internal medicine and emergency medicine were weighted more heavily than other specialties.

The findings from this study can inform assessment practices by supporting the use of COMAT for osteopathic medical schools that do not administer COMAT.