Advances in deep reinforcement learning enable better predictions of human behavior in time-continuous tasks

Area of Science:

• Cognitive Science
• Artificial Intelligence
• Neuroscience

Background:

• Modeling human behavior in complex, time-continuous environments is challenging.
• Deep Q-networks (DQNs) offer a method to link high-dimensional stimuli to motor responses.
• The impact of recent DQN advancements on human behavior modeling remains unclear.

Purpose of the Study:

• To investigate if advanced Deep Reinforcement Learning (RL) models improve the prediction of human motor responses.
• To compare the predictive accuracy of different DQN architectures (baseline, Ape-X, SEED) on human behavioral data.
• To assess the influence of temporal resolution on prediction accuracy in time-continuous tasks.

Main Methods:

• Recorded motor responses from 23 human participants playing three arcade games.
• Utilized stimulus features from baseline DQN, Ape-X, and SEED models as predictors.
• Fitted DQN response probabilities to human motor responses using linear models and analyzed temporal smoothing effects.

Main Results:

• All three DQN models predicted human behavior significantly above chance level.
• The SEED model, incorporating dueling, double Q-learning, and LSTM, showed superior prediction accuracy across all games.
• SEED demonstrated enhanced capability in modeling human behavior at a fine-grained temporal scale compared to baseline and Ape-X.

Conclusions:

• Advances in deep RL, particularly complex models like SEED, enhance the modeling of human behavior in dynamic environments.
• This approach offers a valuable complement to traditional experimental methods, enabling analysis at finer temporal resolutions.
• Future research can leverage sophisticated RL agents to further understand human decision-making and motor control.