A quasi-Newton acceleration for high-dimensional optimization algorithms.

Area of Science:

• Statistics
• Computational Statistics
• Optimization

Background:

• Maximum likelihood estimation (MLE) algorithms like EM and MM often exhibit slow convergence.
• This slow convergence hinders applications in high-dimensional fields such as data mining, genomics, and imaging.
• Existing acceleration techniques are often inadequate for complex, large-parameter models.

Purpose of the Study:

• To introduce a novel quasi-Newton acceleration scheme for statistical estimation algorithms.
• To address the limitations of slow convergence in EM and MM algorithms.
• To provide an efficient alternative to existing acceleration methods like SQUAREM.

Main Methods:

• Development of a new quasi-Newton acceleration scheme.
• Comparison of the new method's performance against SQUAREM on test problems.
• Evaluation of computational increments and storage requirements.

Main Results:

• The proposed quasi-Newton scheme demonstrates competitive or superior performance compared to SQUAREM.
• The acceleration method requires only modest increases in computation per iteration and overall storage.
• Effective acceleration of slow-converging statistical algorithms.

Conclusions:

• The new quasi-Newton acceleration scheme offers an efficient solution for accelerating slow statistical estimations.
• This method is suitable for high-dimensional problems where traditional algorithms falter.
• The approach provides a practical alternative to existing acceleration techniques with comparable or better results.