An evaluation of multiple feed-forward networks on GPUs.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Graphics Processing Units (GPUs) have evolved from graphics rendering to general-purpose computation.
• GPUs offer high performance at a low cost, making them ideal for data-parallel tasks.
• Parallel implementations of machine learning algorithms are essential for complex, demanding problems.

Purpose of the Study:

• To present a GPU parallel implementation of the Back-Propagation (BP) and Multiple Back-Propagation (MBP) algorithms.
• To describe the necessary GPU kernels for efficient parallel computation.
• To develop a preliminary GPU-based Autonomous Training System (ATS) for automated NN solution discovery.

Main Methods:

• Developed GPU kernels for BP and MBP algorithms.
• Implemented parallel computation strategies on GPUs.
• Benchmarked performance against traditional hardware implementations.
• Integrated algorithms into an Autonomous Training System (ATS).

Main Results:

• Achieved significantly faster training times for Neural Networks (NNs) compared to traditional hardware.
• Demonstrated improved performance due to maximized floating-point throughput and memory bandwidth on GPUs.
• Successfully developed a preliminary GPU-based Autonomous Training System (ATS).

Conclusions:

• GPU parallel implementations of BP and MBP algorithms offer substantial speedups and performance gains.
• GPUs are highly effective for accelerating machine learning tasks, particularly Neural Network training.
• The developed ATS shows promise for automating the search for high-quality NN solutions.