A New Concept of Multiple Neural Networks Structure Using Convex Combination

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Conventional neural network (NN) structures can face challenges with training convergence and optimal performance.
• Leveraging collective intelligence from multiple models is a promising area for improving NN efficiency.

Purpose of the Study:

• To introduce and evaluate a novel convex-combined multiple neural network (NN) structure.
• To demonstrate improved training convergence and test accuracy compared to conventional NN architectures.

Main Methods:

• Proposed a convex-combined architecture integrating information from multiple NNs.
• Conducted experiments using recurrent NNs and convolutional NNs.
• Evaluated performance on semantic frame parsing (spoken language understanding - SLU) using the ATIS dataset and handwritten digit recognition using the MNIST dataset.

Main Results:

• The new NN structure achieved 4x-8x faster training convergence.
• The approach demonstrated similar or superior test accuracy compared to conventional NN structures.
• Consistent performance improvements were observed across both SLU and digit recognition tasks.

Conclusions:

• The convex-combined multiple NN structure offers significant advantages in training efficiency.
• This collective information approach is effective for improving deep learning model performance.
• The proposed structure is a viable alternative for tasks requiring fast and accurate model training.