A Hybrid of Generative and Discriminative Models Based on the Gaussian-Coupled Softmax Layer

Area of Science:

• Machine Learning
• Artificial Intelligence
• Deep Learning

Background:

• Generative models offer benefits like unsupervised data access and calibrated confidence for classification.
• Discriminative models excel in performance and have simpler structures and algorithms.
• A gap exists in models that synergize the strengths of both generative and discriminative approaches.

Purpose of the Study:

• To propose a novel method for training a hybrid discriminative-generative model within a single neural network.
• To develop a unified architecture that leverages the advantages of both generative and discriminative learning paradigms.
• To enhance classification tasks through a model capable of estimating both data and class posterior distributions.

Main Methods:

• Introduction of a Gaussian-coupled softmax layer, a novel component for neural networks.
• Embedding the Gaussian-coupled softmax layer into a neural network classifier.
• Utilizing the layer to jointly estimate the input data distribution and class posterior probabilities.

Main Results:

• The proposed hybrid model successfully integrates generative and discriminative characteristics.
• The model demonstrates the ability to estimate both data and class posterior distributions.
• The hybrid approach shows applicability in semi-supervised learning scenarios.
• The method contributes to improved confidence calibration in classification tasks.

Conclusions:

• The developed hybrid model offers a synergistic approach to classification by combining generative and discriminative strengths.
• The Gaussian-coupled softmax layer is a key innovation enabling joint distribution estimation.
• The proposed method advances semi-supervised learning and enhances model confidence calibration.