Guaranteed satisficing and finite regret: Analysis of a cognitive satisficing value function

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning

Background:

• Reinforcement learning (RL) algorithms face challenges in solving complex, realistic tasks within practical timeframes.
• Traditional RL focuses on finding optimal actions, which can be computationally intensive and time-consuming.
• Satisficing strategies, aiming for 'good enough' solutions above an aspiration level, offer an alternative.

Purpose of the Study:

• To introduce a novel mathematical model, risk-sensitive satisficing (RS), for efficient reinforcement learning.
• To implement a satisficing strategy integrating risk-averse and risk-prone attitudes within a greedy policy.
• To evaluate the performance of RS on K-armed bandit problems, a fundamental RL task.

Main Methods:

• Developed the risk-sensitive satisficing (RS) mathematical model.
• Integrated risk-averse and risk-prone attitudes under a greedy policy for satisficing.
• Applied and analyzed the RS model on K-armed bandit problems, proving theoretical propositions.

Main Results:

• Proved that the RS model guarantees finding an action exceeding the defined aspiration level.
• Demonstrated that RS has a finite upper bound on regret (expected loss) when the aspiration level is optimally set.
• Numerical simulations confirmed theoretical results and showed competitive performance against other algorithms.

Conclusions:

• The risk-sensitive satisficing (RS) model provides an effective approach for reinforcement learning tasks where finding optimal solutions is challenging.
• RS offers a practical alternative by efficiently identifying actions that meet or exceed performance expectations.
• The theoretical guarantees and empirical validation suggest RS's utility in complex, real-world RL applications.