The EM algorithm and medical studies: a historical link

Anderson Gray McKendrick's 1926 paper, 'Applications of mathematics to medical problems', was the earliest reference cited in Dempster et al.'s 1977 paper that defined and popularized the EM algorithm. McKendrick's paper was prominently featured by Joseph Oscar Irwin in his 1962 inaugural address as the President of the Royal Statistical Society (in the UK), entitled 'The place of mathematics in medical and biological statistics'. The link of McKendrick's work to the EM algorithm is due to an improvement made by Irwin on a novel method McKendrick used for estimating an infection rate when the observed data do not distinguish between those individuals who are not susceptible to the infection and those who are susceptible, but do not develop symptoms. This article examines this link, along the way illustrating the central ideas underlying the EM algorithm as well as its properties; the examination also suggests a profiling strategy for speeding up EM, which may be worthy of general investigation. McKendrick's data on an epidemic of cholera are used for illustration and to compare EM with Irwin's method as well as the Newton-Raphson algorithm. Issues beyond computation are also discussed whenever appropriate.