Parallel growing and training of neural networks using output parallelism

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neural Networks

Background:

• Designing architectures for large-scale real-world applications requires efficient and automated methods.
• Traditional approaches may lack flexibility or require extensive prior knowledge.

Purpose of the Study:

• To propose a simple neural-network task decomposition method based on output parallelism.
• To enable automatic and efficient architecture design for large-scale applications.

Main Methods:

• Decomposing problems into subproblems, each using the whole input and a fraction of the output vector.
• Decoupling hidden structures and training modules in parallel on processing elements.
• Utilizing a constructive learning algorithm without requiring prior decomposition knowledge.

Main Results:

• Demonstrated feasibility and validity through benchmark implementations.
• Showcased reduction in computational time and increased learning speed.
• Improved generalization accuracy for both classification and regression problems.

Conclusions:

• The proposed output parallelism method offers an effective strategy for neural network task decomposition.
• This approach facilitates efficient, scalable, and accurate solutions for complex machine learning problems.