Algebraic Approach to Maximum Likelihood Factor Analysis

In maximum likelihood factor analysis, we need to solve a complicated system of algebraic equations, known as the normal equation, to get maximum likelihood estimates (MLEs). Since this equation is difficult to solve analytically, its solutions are typically computed with continuous optimization methods, such as the Newton-Raphson method. With this procedure, however, the MLEs are dependent on initial values since the log-likelihood function is highly non-concave. Particularly, the estimates of unique variances can result in zero or negative, referred to as improper solutions; in this case, the MLE can be severely unstable. To delve into the issue of the instability, we algebraically compute all candidates for the MLE. We provide an algorithm based on algebraic computations that is carefully designed for maximum likelihood factor analysis. To be specific, Gröbner bases are employed, powerful tools to get simplified sub-problems for given systems of algebraic equations. Our algebraic algorithm provides the MLE independent of the initial values. While computationally demanding, our algebraic approach is applicable to small-scale problems and provides valuable insights into the characterization of improper solutions. For larger-scale problems, we provide numerical methods as practical alternatives to the algebraic approach. We perform numerical experiments to investigate the characteristics of the MLE with our two approaches.