Learning in Deep Radial Basis Function Networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Radial Basis Function (RBF) networks are typically single-layered due to perceived instability in deeper architectures.
• Existing universal approximation theorems support single-layered RBF networks, limiting exploration of deeper models.
• Deep neural networks, including Convolutional Neural Networks (CNNs), have demonstrated significant success across various tasks.

Purpose of the Study:

• To demonstrate the feasibility and effectiveness of designing stable, multi-layered RBF network architectures.
• To develop efficient learning schemes for deep RBF networks.
• To compare the performance of deep RBF networks against established deep learning models like CNNs.

Main Methods:

• Introduced a novel initialization scheme for deep RBF networks using k-means clustering and covariance estimation.
• Leveraged convolutional operations to accelerate Mahalanobis distance calculations in a partially connected manner, inspired by CNNs.
• Evaluated the proposed deep RBF network approach on image classification and speech emotion recognition datasets.

Main Results:

• Deep RBF networks were successfully designed with efficient learning schemes.
• The proposed methods enabled stable training of multi-layered RBF architectures.
• Deep RBF networks achieved performance comparable to state-of-the-art deep neural networks like CNNs on tested tasks.

Conclusions:

• Deeper RBF network architectures can be effectively implemented and trained.
• The developed initialization and convolutional acceleration techniques contribute to stable and efficient deep RBF learning.
• Deep RBF networks present a viable alternative to other deep learning architectures for complex tasks such as image classification and speech emotion recognition.