An algorithm for learning without external supervision and its application to learning control systems.

Area of Science:

• Control Engineering
• Machine Learning
• Stochastic Systems

Background:

• Controlling discrete stochastic plants presents challenges due to inherent randomness and environmental interactions.
• Existing control methods may not adapt effectively to dynamic plant-environment situations.

Purpose of the Study:

• To propose a novel online learning controller algorithm for discrete stochastic plants.
• To demonstrate the algorithm's ability to converge towards optimal control strategies.
• To provide a robust framework for adaptive control in uncertain environments.

Main Methods:

• An algorithm utilizing two transformations (T1: ordering, T2: learning) to modify subjective action probabilities.
• Estimation of performance indexes for allowable actions.
• Consideration of both discrete and continuous features, employing the Potential Function Method for continuous cases.

Main Results:

• The algorithm proves that the subjective probability of selecting the optimal action approaches one with probability one for any observed event.
• The optimized performance index is the conditional expectation of instantaneous performance evaluations.
• Computer simulations demonstrate the algorithm's effectiveness compared to a linear reinforcement scheme.

Conclusions:

• The proposed online learning controller algorithm effectively adapts to discrete stochastic plants.
• The method guarantees convergence to optimal control policies in dynamic environments.
• This approach offers a significant advancement in adaptive control design for stochastic systems.