Self-organized Operational Neural Networks with Generative Neurons

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Conventional Convolutional Neural Networks (CNNs) suffer from homogeneity and a limited linear neuron model.
• Operational Neural Networks (ONNs) offer heterogeneity but face computationally demanding operator search and limited adaptability.
• Current ONNs rely on fixed operator libraries, risking performance degradation if optimal operators are absent.

Purpose of the Study:

• To introduce Self-organized Operational Neural Networks (Self-ONNs) for enhanced network diversity and computational efficiency.
• To enable generative neurons that adapt nodal operators during training, eliminating the need for a predefined operator set.
• To develop an effective error back-propagation method for Self-ONNs.

Main Methods:

• Proposed Self-organized ONNs (Self-ONNs) with generative neurons capable of optimizing nodal operators.
• Implemented an error back-propagation mechanism tailored for the operational layers of Self-ONNs.
• Conducted experiments on four challenging problems to evaluate Self-ONNs against conventional ONNs and CNNs.

Main Results:

• Self-ONNs demonstrated superior learning capability compared to conventional ONNs and CNNs.
• The proposed method significantly improved computational efficiency.
• Adaptive nodal operators led to greater network diversity and performance.

Conclusions:

• Self-ONNs offer a more effective and efficient approach to deep learning by enabling dynamic operator adaptation.
• The Self-ONN architecture overcomes the limitations of fixed operator sets and computationally intensive search methods.
• This research paves the way for more diverse and performant neural network architectures.