Weighted Error Entropy-Based Information Theoretic Learning for Robust Subspace Representation.

Area of Science:

• Machine Learning
• Computer Vision
• Information Theory

Background:

• Traditional representation learning often assumes noise is independent and identically distributed (i.i.d.) and Gaussian, which limits applicability in practical scenarios.
• Real-world noise, such as in face representation, is often structural (e.g., due to illumination or occlusion) and violates i.i.d. or Gaussian assumptions.

Purpose of the Study:

• To propose a more realistic noise model for robust representation learning.
• To develop an information-theoretic learning framework that leverages this new noise model for improved performance.

Main Methods:

• Devised a generic noise model: independent and piecewise identically distributed (i.p.i.d.) model, characterized by a union of distributions.
• Developed a novel information-theoretic learning framework using a minimum weighted error entropy criterion.
• Applied the proposed scheme to subspace clustering and image recognition.

Main Results:

• The i.p.i.d. model effectively describes complex, non-i.i.d. noise and includes the i.i.d. model as a special case.
• The proposed learning method demonstrates superior robustness against diverse noise types.
• Significant performance gains were observed in subspace clustering and image recognition compared to existing methods.

Conclusions:

• The i.p.i.d. noise model offers a more accurate representation of practical data noise.
• The developed information-theoretic framework provides enhanced robustness and performance in representation learning tasks.
• The approach shows promise for real-world applications like image recognition and data clustering.