Goal-oriented inference of environment from redundant observations

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Reinforcement learning optimizes decision-making for goals like reward maximization.
• Partially Observable Markov Decision Processes (POMDPs) present challenges due to incomplete state information.
• Real-world environments contain irrelevant data, complicating traditional POMDP methods.

Purpose of the Study:

• To develop an efficient goal-oriented reinforcement learning method for POMDPs with redundant observations.
• To address the ineffectiveness of conventional POMDP methods in environments with irrelevant states.
• To propose a novel approach using Redundantly Observable Markov Decision Processes (ROMDPs).

Main Methods:

• Introduced a method to learn state transition rules among reward-related "core states" from redundant observations.
• Gradually expanded the transition diagram by adding core states until an optimal strategy consistent with the Bellman equation was achieved.
• Focused on core states, excluding irrelevant observations, to improve efficiency and explainability.

Main Results:

• The proposed ROMDP-based method demonstrated superior performance compared to conventional POMDP methods.
• The model achieved an optimal behavioral strategy consistent with the Bellman equation.
• The focus on core states resulted in a highly explainable inference model.

Conclusions:

• The novel method efficiently learns optimal strategies in complex, partially observable environments.
• The approach offers high explainability and is suitable for online learning due to reduced memory consumption and faster learning speeds.
• This work advances reinforcement learning techniques for real-world applications with noisy and incomplete data.