A data augmentation approach for a class of statistical inference problems.

Area of Science:

• Statistical inference
• Optimization algorithms
• Computational statistics

Background:

• Statistical inference problems often involve complex models with hidden variables.
• Existing methods like Expectation-Maximization (EM) provide iterative structures but may lack systematic surrogate function generation.
• Maximum Likelihood (ML) and Maximum a Posteriori (MAP) estimations are common but challenging with latent variables.

Purpose of the Study:

• To introduce a novel algorithm for a class of statistical inference problems.
• To reformulate inference as an optimization procedure using surrogate functions.
• To provide a systematic method for constructing these surrogate functions, particularly for problems with hidden variables.

Main Methods:

• The algorithm reformulates statistical inference as an optimization problem.
• It utilizes surrogate (auxiliary) functions generated systematically.
• The approach is inspired by the MM algorithm and the iterative nature of the Expectation-Maximization (EM) algorithm.

Main Results:

• The algorithm effectively handles hidden variables in Maximum Likelihood (ML) and Maximum a Posteriori (MAP) estimation.
• It is applicable to Instrumental Variables, Regularized Optimization, and Constrained Optimization problems.
• Numerical examples demonstrate the advantages and effectiveness of the proposed approach.

Conclusions:

• The proposed algorithm offers a systematic procedure for constructing surrogate functions.
• This facilitates the resolution of statistical inference problems involving hidden variables.
• The method shows practical benefits in various complex estimation and optimization scenarios.