Parallel orthogonal deep neural network.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Ensemble learning methods enhance model performance by combining multiple base models.
• The effectiveness of ensembles relies on the diversity among base models.
• Current ensemble methods lack mechanisms to guarantee or control model diversity.

Purpose of the Study:

• To propose a novel parallel orthogonal deep learning architecture.
• To enforce diversity among ensemble members by design using an orthogonality constraint.
• To improve classification performance by ensuring base model dissimilarity.

Main Methods:

• A parallel architecture with multiple deep neural networks is introduced.
• An orthogonality constraint is imposed on the outputs of base models at each parallel layer.
• Gram-Schmidt orthogonalization is applied to enforce diversity.

Main Results:

• The proposed method achieves high diversity between models.
• Models exhibit different error patterns across the feature space.
• Models show varying levels of decision uncertainty.
• Experimental results demonstrate superior classification performance compared to standard methods.

Conclusions:

• The orthogonal deep learning architecture effectively enforces model diversity.
• Enforced diversity leads to improved classification performance.
• This approach offers a mechanism to guarantee diversity in ensemble learning.