Fully complex conjugate gradient-based neural networks using Wirtinger calculus framework: Deterministic convergence...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Conjugate gradient (CG) methods are effective for neural network training due to low memory needs and fast convergence.
• Complex-valued neural networks (CVNNs) offer advantages in signal processing and pattern recognition but pose training challenges.

Purpose of the Study:

• To propose an efficient conjugate gradient (CG) method tailored for training fully complex-valued neural network (CVNN) models.
• To enhance training performance through improved descent direction and adaptive learning rates.

Main Methods:

• Developed a novel CG method utilizing a fine-tuned conjugate coefficient to ensure a sufficient descent direction.
• Implemented a generalized Armijo search to determine optimal learning rates dynamically in each iteration, replacing fixed rates.
• Applied the Wirtinger differential operator for complex-valued network training.

Main Results:

• Demonstrated weak and strong convergence properties, proving that gradient norms approach zero and weight sequences converge to optimal points.
• Achieved enhanced training performance in terms of speed and convergence compared to standard methods.
• Validated the method's effectiveness and rationality through four simulations on regression and classification tasks.

Conclusions:

• The proposed conjugate gradient method offers an efficient and robust approach for training complex-valued neural networks.
• The combination of a fine-tuned coefficient and adaptive learning rates significantly improves training dynamics.
• The method shows strong potential for practical applications in complex-valued deep learning.