Micro-Supervised Disturbance Learning: A Perspective of Representation Probability Distribution.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Existing representation learning methods based on Euclidean distance exhibit instability.
• The scarcity and high cost of labels necessitate more expressive representation learning methods requiring fewer labels.

Purpose of the Study:

• To introduce a small-perturbation ideology into representation learning models based on probability distributions.
• To develop novel models that fine-tune representation distributions using minimal labeled data.

Main Methods:

• Introduced positive small-perturbation information (SPI) dependent on two labels per cluster.
• Proposed Micro-supervised Disturbance Gaussian-binary RBM (Micro-DGRBM) and Micro-supervised Disturbance RBM (Micro-DRBM) models.
• Utilized Kullback-Leibler (KL) divergence to minimize SPI within clusters and maximize it between clusters during Contrastive Divergence (CD) learning.
• Developed a deep Micro-supervised Disturbance Learning (Micro-DL) framework.

Main Results:

• The deep Micro-DL architecture demonstrated superior performance compared to baseline methods, shallow models, and other deep frameworks.
• The proposed models effectively stimulated representation probability distributions using SPI.
• Minimizing KL divergence of SPI within clusters promoted similarity, while maximizing it between clusters enforced dissimilarity in representation distributions.

Conclusions:

• The Micro-DL framework offers a robust and effective approach to representation learning, particularly in low-label scenarios.
• The small-perturbation ideology enhances the stability and expressiveness of representation learning models.
• The proposed models show significant improvements in clustering tasks.