Tracking strategy changes using machine learning classifiers.

Area of Science:

• Cognitive science
• Human-computer interaction
• Machine learning

Background:

• High performers in complex tasks often use superior strategies, yet understanding strategy formation and change is limited.
• Identifying cognitive strategies from observable behavior in tasks with many possibilities presents a significant research challenge.

Purpose of the Study:

• To develop and evaluate algorithms capable of identifying the task features utilized in complex task strategies.
• To compare the efficacy of machine learning-based algorithms against a task-specific algorithm in strategy identification.

Main Methods:

• Three algorithms were developed: a support vector machine, a decision tree algorithm (both task-general machine learning classifiers), and a task-specific algorithm.
• Simulated data from various strategies within a complex task were used to test the algorithms' ability to identify underlying strategy features.
• Algorithm performance was assessed based on strategy complexity, data quantity, and frequency of strategy changes.

Main Results:

• Task-general machine learning classifiers (support vector machine, decision tree) demonstrated superior performance compared to the task-specific algorithm.
• The effectiveness of algorithms in recovering strategies was contingent on strategy complexity relative to available performance data.
• Despite performance degradation with frequent strategy shifts, algorithms successfully tracked strategy changes in simulated and human participant data.

Conclusions:

• Machine learning approaches offer a promising method for tracking strategy exploration in complex tasks.
• This algorithmic approach can facilitate the development of more robust theories regarding how individuals discover and adopt effective strategies.
• Further research using these algorithms can illuminate the cognitive processes underlying strategy formation and adaptation.