A Cornucopia of Maximum Likelihood Algorithms

Classroom expositions of maximum likelihood estimation (MLE) rely on traditional calculus methods to construct analytic solutions. This creates in students a false sense of the ease with which MLE problems can be attacked. In a nod to reality, some teachers mention and apply Newton's method, Fisher scoring, and the expectation-maximization (EM) algorithm. Although preferable to leaving students in a state of ignorance, such brief expositions ultimately fail to expose the full body of relevant techniques. Some of these techniques extend more readily to high-dimensional data problems than Newton's method and scoring. The current paper emphasizes block ascent and descent, profile likelihoods, the minorization-maximization (MM) principle, and their creative combination. These themes are put to work in readable Julia code to solve several MLE problems.