A Cornucopia of Maximum Likelihood Algorithms

Area of Science:

• Statistics
• Computational Statistics
• Numerical Optimization

Background:

• Traditional classroom teaching of maximum likelihood estimation (MLE) uses calculus, which can oversimplify problem-solving.
• Existing supplementary methods like Newton's method, Fisher scoring, and EM algorithm offer limited scope, especially for high-dimensional data.
• There's a need for more robust and scalable techniques in statistical inference education.

Purpose of the Study:

• To present advanced computational techniques for maximum likelihood estimation (MLE).
• To demonstrate the application of these methods to solve complex MLE problems.
• To provide educators and students with practical alternatives to traditional calculus-based approaches.

Main Methods:

• Emphasis on block ascent and descent algorithms.
• Application of profile likelihoods for model simplification.
• Integration of the minorization-maximization (MM) principle.
• Creative combination of these techniques.
• Implementation using readable Julia code.

Main Results:

• Demonstration of how advanced methods can be practically applied to MLE problems.
• Illustrates the effectiveness of block ascent, profile likelihoods, and MM principles.
• Provides a computational framework in Julia for solving challenging estimation tasks.

Conclusions:

• Advanced techniques like block ascent, profile likelihoods, and MM are crucial for modern MLE, especially with high-dimensional data.
• These methods offer a more realistic and powerful approach compared to traditional calculus-based solutions.
• The presented Julia code facilitates the learning and application of these advanced statistical inference techniques.