Evidence-based evolution of the high stakes postgraduate intensive care examination in Australia and New Zealand

The fellowship examination for intensive care medicine in Australia and New Zealand, first held in 1979, has undergone four major periods of development and change since inception. These periods are characterised as: 1. 1979 to 1996--initiation and establishment of the exam as a relevant and comprehensive assessment process for a new specialty. 2. 1997 to 2001--revision to increase breadth of coverage, increase reliability for a growing number of candidates and ensure that each candidate received the same exam: Expansion: to incorporate assessment of CanMEDS skills (including communication, procedures and professional qualities). Lengthening: to increase the number of exposures, to ensure reliability. Quarantining of candidates: to allow the provision of a similar exam for each candidate. 3. 2002 to 2006--increasing emphasis on examiner training, standard setting and increasing feedback to candidates to improve the educational experience and guide exam preparation. Blueprinting of questions to maintain validity. 4. 2008 onwards--logistic revision to ensure feasibility for a rapidly growing number of candidates and refinement to apply modem standard setting and quality control. The exam has been regarded as a 'tough but fair' assessment in its 30 years of existence and the committee overseeing its development has aimed to continually review the process to maintain those qualities as well as reliability, validity and feasibility. The increasing number of candidates has allowed accumulation of usable statistics but has tested the feasibility of running such a labour intensive exam. To date, there have been 800 presentations to the exam with 498 successful candidates.