SVM-based tree-type neural networks as a critic in adaptive critic designs for control

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Machine Learning

Background:

• Adaptive critic design (ACD) is a control approach that uses neural networks to approximate system dynamics and optimal control policies.
• Action-dependent heuristic dynamic programming (ADHDP) is a specific ACD method that learns control policies based on actions taken.
• Support Vector Machines (SVMs) are powerful supervised learning models used for classification and regression tasks.

Purpose of the Study:

• To develop and demonstrate an improved control strategy using adaptive critic design (ACD) and support vector machines (SVMs).
• To address the challenge of learning from imbalanced binary classification data, specifically failure states in control systems.
• To showcase the capability of the trained controller in handling unforeseen situations.

Main Methods:

• Implementation of the action-dependent heuristic dynamic programming (ADHDP) method for control.
• Utilization of a least squares support vector machine (SVM) regressor for control action generation.
• Employment of an SVM-based tree-type neural network (NN) as the critic, capable of dynamic neuron addition for learning imbalanced data.
• Retraining of the critic and action generation model using failure data post-occurrence.

Main Results:

• The SVM-based tree-type NN successfully learned from imbalanced binary classification data, overcoming limitations of conventional multilayer feedforward NNs.
• The developed controller demonstrated the ability to adapt and handle unforeseen situations effectively.
• The retraining mechanism using failure data improved the controller's performance after failures.

Conclusions:

• The proposed ADHDP approach with SVM-based components offers a robust solution for control problems with imbalanced failure data.
• The adaptive nature and learning capability of the SVM-based tree-type NN critic are key to handling complex classification tasks.
• This method enhances the reliability and adaptability of control systems, particularly in scenarios prone to failures.