Deep supervised learning with mixture of neural networks.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computer Science

Background:

• Deep Neural Networks (DNNs) excel at classification but struggle with heterogeneous data distributions or unobserved factors.
• Training a single DNN model becomes challenging when data lacks uniformity, limiting performance in complex classification tasks.

Purpose of the Study:

• To introduce Mixture of Deep Neural Networks (MoNNs), a novel supervised approach for classification tasks involving heterogeneous data.
• To enhance classification performance by effectively handling data with varying distributions and latent factors.

Main Methods:

• Proposed MoNNs utilize a gating network, implemented as a neural network, to partition heterogeneous data into homogeneous subsets.
• Multiple Deep Neural Networks (DNNs) function as local expert models, processing data within their respective homogeneous components.
• The Expectation-Maximization (EM) algorithm is employed for optimizing the MoNNs model parameters.

Main Results:

• MoNNs demonstrated superior performance compared to existing methods in classification tasks.
• Experimental validation included the determination of diabetes, differentiation of benign and malignant breast cancer, and handwriting recognition.
• The proposed MoNNs effectively addressed data heterogeneity, leading to improved classification outcomes.

Conclusions:

• MoNNs provide a robust solution for classification problems characterized by data heterogeneity.
• The mixture model approach, combining a gating network with multiple DNNs, significantly enhances classification accuracy.
• This methodology offers a promising direction for improving DNN performance in real-world, complex datasets.