Longitudinal reliability of the Royal Australian College of General Practitioners certification examination

The examination for Fellowship of the Royal Australian College of General Practitioners is a long-standing examination that is regarded as a valid and comprehensive assessment of competence for general practice in Australia. Income and employment opportunities now depend on passing this battery of eight subtests. This paper describes the structures of the examination and reports an analysis of the reliability and efficiency of the examinations, based on data from five consecutive examinations. Results indicate that the examination achieves acceptable overall reliability, although suggestions for changes to improve reliability are made. A decision on implementing any changes must take into consideration any consequent changes in validity.