A Generic Self-Supervised Framework of Learning Invariant Discriminative Features.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• Self-supervised learning (SSL) generates invariant representations without human annotations but requires data-specific transformations.
• Existing SSL frameworks are often tailored to specific data types, limiting their generalizability.
• Autoencoders (AEs) are general but primarily focus on dimensionality reduction, not invariant representation learning.

Purpose of the Study:

• To propose a generic self-supervised learning framework for learning invariant representations.
• To overcome the limitations of data-specific transformations in current SSL methods.
• To develop a robust representation learning strategy applicable across diverse data modalities.

Main Methods:

• Introduced a constrained self-labeling assignment process to prevent degenerate solutions.
• Replaced data-specific transformations with a self-transformation mechanism derived via adversarial training.
• Designed a training objective using contrastive learning, integrating self-labeling and self-transformation.

Main Results:

• The proposed generic SSL framework demonstrated superior performance compared to state-of-the-art AE-based representation learning methods.
• Experiments on visual, audio, text, and mass spectrometry data validated the method's effectiveness.
• The approach proved robust in identifying patterns across diverse datasets.

Conclusions:

• The proposed generic self-supervised learning framework effectively learns invariant representations.
• The self-transformation mechanism offers a versatile approach applicable to various data types.
• This method provides a robust and generalizable solution for representation learning.