Two-level group convolution.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Group convolution reduces computation time in convolutional neural networks but suffers performance degradation with many groups.
• Standard convolution is computationally intensive, dominating neural network training time.

Purpose of the Study:

• To propose a new convolution methodology, "two-level" group convolution, that maintains performance with an increased number of groups.
• To develop a method suitable for multi-GPU parallel computation and robust against performance degradation.

Main Methods:

• Interpreting group convolution as a one-level block Jacobi approximation from numerical analysis.
• Introducing a coarse-level structure to promote intergroup communication without creating bottlenecks.
• Ensuring additional work from the coarse-level structure is efficiently processed in distributed memory systems.

Main Results:

• Demonstrated robustness of the two-level group convolution method with respect to the number of groups.
• Verified efficient processing of the coarse-level structure in distributed memory systems.
• Numerical results confirm the proposed method's superiority over existing group convolution approaches.

Conclusions:

• The proposed two-level group convolution effectively addresses the performance degradation issue associated with increasing group numbers.
• This method offers significant improvements in execution time, memory efficiency, and overall performance for deep learning models.
• The approach is well-suited for multi-GPU parallel computation, enhancing scalability.