Improving deep convolutional neural networks with mixed maxout units

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep Convolutional Neural Networks (CNNs) utilize maxout units for feature extraction.
• Existing maxout units face limitations in feature mapping subspace pooling and delivering non-maximal features.

Purpose of the Study:

• To introduce a novel 'mixout unit' as an advancement over traditional maxout units.
• To enhance the feature pooling and learning capabilities of CNNs.
• To improve classification performance in deep learning models.

Main Methods:

• Developed a mixout unit by calculating exponential probabilities of feature mappings from diverse convolutional transformations.
• Incorporated Bernoulli distribution to balance maximum and expected values within the feature mapping subspace.
• Designed a simple model for verifying mixout unit pooling and a Mixout-units-based Network-in-Network (NiN) for feature learning analysis.

Main Results:

• The proposed mixout units demonstrate improved pooling abilities compared to standard maxout units.
• Mixout-based models exhibit enhanced feature learning capabilities.
• Preliminary results indicate better classification performance with mixout models.

Conclusions:

• Mixout units represent a significant improvement over traditional maxout units in CNNs.
• The novel approach enhances feature representation and pooling, leading to superior model performance.
• Further research into mixout units can advance deep learning architectures.