Learning Dynamics and Control of a Stochastic System under Limited Sensing Capabilities

Area of Science:

• Control Theory
• Machine Learning
• Stochastic Systems

Background:

• Systems operating under uncertainty require runtime sensing and control strategies for safe behavior.
• Reconstructing unknown control strategies from system state and actions is challenging due to sensing limitations and noise.

Purpose of the Study:

• To develop an optimal control action selection method for stochastic systems with unknown dynamics and unknown control strategies.
• To address challenges posed by limited trajectory data, noisy state observations, and high failure costs preventing online exploration.

Main Methods:

• Training an Input-Output Hidden Markov Model (IO-HMM) as a generative stochastic model for POMDP state dynamics.
• Utilizing a novel optimization objective to mitigate issues related to model mis-specification.
• Applying the methodology to a failure avoidance scenario in a multi-component system.

Main Results:

• The proposed approach effectively trains an IO-HMM for systems with unknown dynamics and control strategies.
• The novel optimization objective improves model suitability for control tasks compared to traditional methods.
• Decision-making quality, evaluated by collected rewards, shows improvement over existing literature approaches.

Conclusions:

• The developed methodology offers a robust solution for learning and optimizing control strategies in complex, uncertain systems.
• This approach enhances safety and efficiency in applications like failure avoidance for multi-component systems.
• It provides a viable alternative to traditional methods that often fail due to model mis-specification.