Minimax mutual information approach for independent component analysis

Area of Science:

• Signal Processing
• Information Theory
• Machine Learning

Background:

• Independent Component Analysis (ICA) commonly uses minimum mutual information as a performance measure.
• Existing information-theoretic ICA algorithms include minimum mutual information and maximum output entropy approaches.
• Higher-order cumulant-based methods offer alternative ICA solutions related to these information-theoretic approaches.

Purpose of the Study:

• Propose a new ICA algorithm based on the maximum entropy principle.
• Develop an algorithm optimizing minimum output mutual information using estimated probability density functions (pdfs).
• Introduce the minimax mutual information ICA algorithm, providing an upper bound for output signal mutual information.

Main Methods:

• Utilize the maximum entropy principle for estimating signal distributions.
• Employ exponential family distributions for probability density function (pdf) estimation.
• Solve a constrained entropy maximization problem to approximate source distributions.

Main Results:

• The proposed algorithm achieves minimum output mutual information.
• The method provides an upper bound for the mutual information of output signals.
• Demonstrated a strong relationship between the new algorithm and higher-order cumulant methods for specific constraint functions.

Conclusions:

• The minimax mutual information ICA algorithm offers a novel approach to Independent Component Analysis.
• The algorithm effectively estimates signal distributions using maximum entropy principles.
• This method connects information-theoretic ICA with higher-order cumulant-based techniques.