Adaptive Learning Rate Methods for Complex-Valued Neural Networks.

Area of Science:

• Digital Signal Processing (DSP)
• Machine Learning
• Complex-Valued Neural Networks (CVNNs)

Background:

• Artificial neural networks (ANNs) are widely used in DSP.
• Complex-valued neural networks (CVNNs) offer advantages over real-valued neural networks (RVNNs) in handling complex domain signals, leading to higher accuracy and faster convergence.
• However, CVNNs lack advanced learning techniques compared to RVNNs.

Purpose of the Study:

• To propose adaptive learning rate optimization approaches for CVNNs.
• To extend established adaptive gradient algorithms to the complex domain for CVNNs.
• To analyze the computational complexity and performance of these novel CVNN optimizers.

Main Methods:

• Extension of AdaGrad, RMSProp, AdaMax, AMSGrad, SAMSGrad, Nadam, and DiffGrad to the complex domain.
• Analysis of computational complexities for CVNN architectures using the proposed optimizers.
• Comparative evaluation of mean-squared-error convergence for different adaptive learning rate approaches.

Main Results:

• The proposed adaptive learning rate methods are successfully extended to the complex domain for CVNNs.
• Computational complexities of the novel optimizers are analyzed for CVNNs.
• Performance is evaluated based on mean-squared-error convergence, demonstrating potential improvements.

Conclusions:

• The developed adaptive learning rate approaches enhance the training of CVNNs.
• These methods address the gap in learning techniques for CVNNs, potentially improving their applicability in image processing and telecommunications.
• Further research can explore broader applications and optimizations of these complex-valued adaptive learning algorithms.