Variable three-term conjugate gradient method for training artificial neural networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Optimization Algorithms

Background:

• Artificial neural networks (ANNs) are widely used computational tools.
• Stochastic gradient descent (SGD) and adaptive methods like Adam are common but limited by first-order gradients.
• Higher-order methods face computational costs and Hessian matrix requirements.

Purpose of the Study:

• To propose a novel optimization algorithm, the variable three-term conjugate gradient (VTTCG) method.
• To enhance search direction by approximating the Hessian matrix.
• To improve convergence stability using a variable step size.

Main Methods:

• Developed and implemented the variable three-term conjugate gradient (VTTCG) method.
• Trained ANNs on benchmark image classification and generation datasets.
• Applied VTTCG to train a grasp generation and selection convolutional neural network (GGS-CNN) for robotic grasping in simulation and on a physical robot.

Main Results:

• The VTTCG method demonstrated superior performance compared to SGD, Adam, AMSGrad, and AdaBelief.
• VTTCG achieved improved convergence stability in ANN training.
• Effective application of VTTCG in both image-related tasks and intelligent robotic grasping.

Conclusions:

• The proposed VTTCG method offers a significant improvement over existing optimization algorithms for training ANNs.
• VTTCG provides a computationally efficient and stable approach for complex tasks like robotic grasping.
• This method has broad applicability in various machine learning and engineering domains.