What is the chance all your trainees will pass the next Fellowship exam: A statistician's view

Your department has had a good track record over many years for preparing trainees to successfully sit for the ACEM Fellowship exam. On average the pass rate for your trainees is over 80%. Then, to your dismay, suddenly only two of five of your trainees pass the latest Fellowship exam. Does this anomaly necessitate an urgent review of your department's training programme, or is it just a statistical quirk? Let us suppose you can prepare candidates so that they all have at least an 80% chance of passing. The probability that all five candidates would have passed is 32.8% (or 0.8⁵ ) based on the multiplication rule of probability for independent events. The probability that only two of five passed is 5.1% (or 10 × 0.8² × 0.2³ ) based on the binomial distribution, which is a probability distribution analogous to the normal distribution. The construction of the binomial distribution depends on two parameters: (i) number of candidates sitting ('n'), and (ii) probability of passing for any individual candidate ('P'). The distribution gives the probability that 'x' number of individuals will pass when 'n' number of individuals sit. Thus despite an 80% pass rate historically, the probability that only two of five candidates will pass is not negligible at 5.1%. It is an anomaly, which we may choose not to act on unless it is recurrent, noting it will be expected to occur naturally about one time out of 20. The real challenge is to maintain or increase that individual probability at 80% or higher.