A Workflow for Building Computationally Rational Models of Human Behavior.

Area of Science:

• Cognitive Science
• Artificial Intelligence
• Machine Learning

Background:

• Computational rationality models human behavior as utility maximization under cognitive and environmental constraints.
• Partially Observable Markov Decision Processes (POMDPs) and reinforcement learning (RL) offer powerful frameworks for adaptive behavior modeling.
• Traditional cognitive architectures often rely on ad hoc rules, contrasting with RL-based approaches.

Purpose of the Study:

• To outline a workflow for building computationally rational models using reinforcement learning (RL).
• To address the complexities and pitfalls in integrating cognitive science and machine learning (ML) practices.
• To provide guidance on key decision points in model development.

Main Methods:

• Instantiating computational rationality via POMDPs.
• Applying reinforcement learning (RL) methods to approximate optimal policies for sequential decision-making.
• Integrating psychological assumptions with ML decisions (reward specification, policy optimization, parameter inference, model selection).

Main Results:

• A decade of work has culminated in a structured workflow for developing computationally rational models.
• The workflow addresses the hybrid nature of these models, drawing from both cognitive science and ML.
• Key decision points in model building are identified with discussions on their pros and cons.

Conclusions:

• Building computationally rational models with RL is challenging due to the integration of cognitive and ML principles.
• The proposed workflow aims to mitigate pitfalls and enhance the development of valid and effective cognitive models.
• This approach offers a more principled alternative to ad hoc rules for capturing human adaptive behavior.