Adaptive CL-BFGS Algorithms for Complex-Valued Neural Networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Numerical Optimization

Background:

• Complex-valued neural networks (CVNNs) offer advantages in specific applications.
• The limited-memory BFGS (CL-BFGS) algorithm is effective for training CVNNs.
• Determining the optimal memory size for CL-BFGS is a significant challenge.

Purpose of the Study:

• To propose an adaptive method for optimizing memory size in the CL-BFGS algorithm for CVNN training.
• To enhance the performance and efficiency of CVNN training by dynamically adjusting memory size.

Main Methods:

• An adaptive CL-BFGS (ACL-BFGS) algorithm is developed, allowing memory size to vary during iterations.
• A multistep quasi-Newton method selects an appropriate memory size from a set {1, ..., M}.
• The upper bound M is adjustable based on the moving average of previous memory sizes.
• Multiple memory sizes are utilized to construct the search direction for improved performance.

Main Results:

• The ACL-BFGS algorithm demonstrates efficient application in training CVNNs.
• Experimental results show advantages over previous methods in pattern classification, complex function approximation, and nonlinear channel equalization.
• The adaptive memory size selection improves computational efficiency and algorithm performance.

Conclusions:

• The proposed adaptive CL-BFGS algorithm effectively addresses the challenge of memory size selection in CVNN training.
• Dynamic memory size adjustment and multiple memory size utilization lead to superior performance and efficiency.
• The ACL-BFGS algorithm offers a promising approach for advancing CVNN research and applications.