Use of a formative mock examination in predicting Australian general practice licensure examination performance

The Royal Australian College of General Practitioners (RACGP) administers high-stakes summative licensure examinations for unsupervised independent Australian general practice. Examination failure can have adverse consequences including psychological stress and financial implications. In-training assessments as predictors of summative or licensure exam outcome are frequently administered as early-training stage formative assessments. Assessments more proximate to the time of licensure examinations may also be of utility. The Mock-AKT, a formative assessment, was available to registrars of an Australian GP vocational training organisation 2 months prior to licensure examinations. The Mock-AKT was assessed over 12 months from 2017 to 2018 for predictive ability for two RACGP licencing written examinations: the applied knowledge test (AKT), a multiple-choice question-based examination, and key feature problem (KFP), a short answer-based examination. Repeat validation was completed in 2021. It had robust ability to predict outcomes of the two written licensure examinations, with excellent area under the receiver operating characteristic curves for both examinations (0.86 for AKT, and 0.82 for KFP). Mock-AKT scores were processed and used to provide medical educators with detailed and candidate-specific failure-risk information, informing discussions regarding registrars' progress and suitability to sit examinations. The Australian Mock-AKT programme is likely generalisable to other GP specialist training contexts.