Reinforcement Learning for Production-Based Cognitive Models

Area of Science:

• Cognitive Science
• Artificial Intelligence
• Computational Neuroscience

Background:

• Production-based cognitive models like ACT-R are valuable for simulating sequential decision-making.
• These models often require extensive manual coding, limiting their learnability and cognitive plausibility, especially for complex linguistic tasks.

Purpose of the Study:

• To investigate the application of reinforcement learning (RL) methods for automating production selection and ordering within the ACT-R cognitive architecture.
• To evaluate the performance of various RL algorithms against the standard ACT-R utility learning algorithm.

Main Methods:

• Employed tabular Q-learning and three deep Q-network (DQN) variants.
• Utilized the ACT-R utility learning algorithm as a baseline for comparison.
• Tested algorithms on lexical decision (LD) tasks framed as sequential decision problems.

Main Results:

• RL agents, particularly DQNs, demonstrated effective learning on complex lexical decision tasks, outperforming the ACT-R baseline.
• Tabular Q-learning and DQNs exhibited a trade-off between learning speed, task complexity, and rule noise.
• Function-approximation-based RL methods showed strong generalization capabilities in complex, realistic scenarios.

Conclusions:

• Reinforcement learning offers a promising avenue for enhancing the learnability and automation of production rules in cognitive models.
• The ACT-R subsymbolic system for procedural memory could benefit from integrating advanced RL techniques, especially function-approximation-based approaches.
• RL methods improve the cognitive plausibility and scalability of computational cognitive models.