MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning.

Area of Science:

• Machine Learning
• Computational Statistics
• Optimization

Background:

• Efficient approximation is crucial for large-scale machine learning.
• Existing methods face challenges with high-dimensional data and numerous moments.
• Maximum entropy methods offer a principled approach to approximation.

Purpose of the Study:

• To propose a novel, robust maximum entropy algorithm for efficient approximation.
• To demonstrate the algorithm's capability in handling hundreds of moments.
• To showcase its superiority over existing methods in specific applications.

Main Methods:

• Development of a novel maximum entropy algorithm.
• Equivalence proof relating the algorithm to constrained Bayesian variational inference.
• Application and evaluation in log determinant estimation and Bayesian optimization.

Main Results:

• The proposed maximum entropy algorithm provides computationally efficient approximations.
• The method demonstrates robustness in handling a large number of moments.
• Superior performance compared to existing approaches in the tested applications.

Conclusions:

• The novel maximum entropy algorithm offers a powerful tool for efficient approximation in machine learning.
• The method's equivalence to Bayesian variational inference provides theoretical grounding.
• Demonstrated practical advantages in fast log determinant estimation and Bayesian optimization.