Reducing Data Complexity Using Autoencoders With Class-Informed Loss Functions.

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Machine learning data is increasingly complex due to high dimensionality and difficult class separations.
• Existing data transformation methods often focus on dimensionality reduction, neglecting class label information for complexity reduction.
• A gap exists in techniques specifically designed to reduce data complexity while leveraging class label information.

Purpose of the Study:

• To propose a novel autoencoder-based approach for reducing data complexity by incorporating class labels.
• To develop new feature learners (Scorer, Skaler, Slicer) that utilize class-informed autoencoders.
• To evaluate the effectiveness of this approach as a preprocessing step for binary classification tasks.

Main Methods:

• Developed a class-informed autoencoder architecture where the loss function uses class labels to guide the generation of informative variables.
• Introduced three feature learners: Scorer (Fisher's discriminant ratio), Skaler (Kullback-Leibler divergence), and Slicer (least-squares support vector machines).
• Applied these methods as a preprocessing stage for binary classification on 27 diverse datasets.

Main Results:

• Class-informed autoencoders demonstrated superior performance in reducing data complexity and improving classification accuracy.
• The proposed methods outperformed four other popular unsupervised feature extraction techniques across various complexity and classification metrics.
• Effectiveness was particularly pronounced when the ultimate goal was to utilize the extracted features for classification.

Conclusions:

• Autoencoder-based complexity reduction using class labels offers a significant advancement over traditional feature extraction methods.
• The Scorer, Skaler, and Slicer learners provide effective means to preprocess complex data for enhanced binary classification.
• This approach effectively addresses the challenge of complex datasets by intelligently leveraging available class information.