Changes in Family Medicine Certification Examination Performance in Longitudinal Assessment

The Family Medicine Certification Longitudinal Assessment (FMCLA) is an open book spaced alternative to the traditional 1-day Family Medicine Certification Examination (FMCE). We studied whether FMCLA improves test scores beyond predicted scores.

We used American Board of Family Medicine data from 2008 to 2023, with scores reported on a scale from 200 to 800. We built a predictive model of FMCE performance using linear regression with the 2008 exam cohort. We then applied that model to the 2009 to 2011 cohorts, who could opt for FMCLA beginning in 2019, controlling for choice of exam mode using an inverse probability weight for selecting FMCE.

Our final sample sizes were 9699 for the FMCE predictive model and 12,851 for the FMCLA versus FMCE analysis. Physicians who opted for FMCLA were more likely to be younger, female, US medical graduates, and have lower prior FMCE scores. Adjusted analyses controlling for propensity to select FMCE revealed that FMCLA was associated with an increase of 39 points (95% CI 36.2-42.0) over predicted score with physicians on the high end scoring 100 more points than predicted, while physicians predicted to fail did worse. Modality of exam had no impact near the passing score.

FMCLA was associated with gains in scores over a 1-day exam. This supports cognitive psychology principles such as spaced repetition and immediate feedback that can strengthen long-term knowledge retention. There was little impact of exam delivery method on scores near the passing score, which suggests that important functions of assessing knowledge can be retained in longitudinal assessment.