Padé Neurons for Efficient Neural Models.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Traditional neural networks utilize the McCulloch-Pitts neuron model, a linear model with point-wise non-linear activation.
• Existing advanced neuron models offer stronger non-linearity but lack the comprehensive integration of Paons.

Purpose of the Study:

• Introduce Padé neurons (Paons), a novel, inherently non-linear neuron model inspired by Padé approximants.
• Demonstrate the advantages of Paons, including diverse non-linearity and layer efficiency.
• Showcase Paons as a universal replacement for existing neuron models.

Main Methods:

• Developed the Padé neuron (Paon) model, learning diverse non-linear functions of inputs.
• Integrated Paons into ResNet-based architectures for image super-resolution, compression, and classification tasks.
• Compared Paon-based models against classic neural network counterparts.

Main Results:

• Paon-based neural networks achieved comparable or superior performance to traditional models.
• Models utilizing Paons required significantly fewer layers for enhanced non-linearity.
• Experimental validation confirmed the efficacy and versatility of Paons across various tasks.

Conclusions:

• Padé neurons (Paons) represent a significant advancement in neural network architecture.
• Paons offer improved performance and efficiency, making them a valuable alternative to conventional neuron models.
• The open-sourced implementation facilitates the adoption of Paons in future deep learning research.