Rethinking exploration-exploitation trade-off in reinforcement learning via cognitive consistency

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Reinforcement Learning

Background:

• The exploration-exploitation dilemma is a core challenge in deep reinforcement learning (RL).
• Existing exploration methods often lack sample efficiency.
• Humans maintain cognitive consistency for effective decision-making.

Purpose of the Study:

• To propose a novel framework, Cognitive Consistency (CoCo), for deep reinforcement learning.
• To improve sample efficiency and performance in RL by rethinking the exploration-exploitation trade-off.
• To leverage cognitive consistency principles for more intelligent agent behavior.

Main Methods:

• Developed the Cognitive Consistency (CoCo) framework.
• Utilized a self-imitating distribution correction for optimal policy cognition.
• Implemented pessimistic exploration with inconsistency-minimization objectives inspired by label distribution learning.

Main Results:

• The CoCo framework was validated on standard off-policy RL tasks.
• Maintaining cognitive consistency demonstrably improved sample efficiency.
• Enhanced performance was observed across tested RL tasks.

Conclusions:

• Cognitive consistency offers a promising approach to enhance RL.
• The CoCo framework effectively balances exploration and exploitation.
• This method improves both the speed and quality of learning in RL agents.