A conversation-based process tracing method for use with naturalistic decisions: an evaluation study.

Area of Science:

• Cognitive Psychology
• Decision Science
• Human-Computer Interaction

Background:

• Traditional process tracing methods may not fully capture naturalistic decision-making.
• Existing think-aloud techniques can be artificial.
• Need for methods that integrate concurrent and retrospective data collection.

Purpose of the Study:

• To evaluate a modified Active Information Search (AIS) technique for naturalistic decisions.
• To assess the reactivity and data richness of a conversational process tracing method.
• To explore the utility of this method for understanding consumer choice tasks.

Main Methods:

• Development of a conversational process tracing method based on Active Information Search (AIS).
• Inclusion of spoken responses and think-aloud instructions within a dialogue.
• Collection of concurrent verbal protocols and retrospective post-decision summaries.
• Evaluation of method reactivity, particularly Preliminary Attribute Elicitation.

Main Results:

• The conversational process tracing method demonstrated minimal reactivity compared to standard think-aloud techniques.
• Preliminary Attribute Elicitation showed some evidence of reactivity.
• The method successfully generated both concurrent and retrospective data.
• Descriptive data suggest the method's potential for theoretical insights into decision-making models.

Conclusions:

• The developed conversational process tracing method is a promising tool for studying naturalistic decisions.
• The technique offers a less reactive alternative to traditional methods, enhancing ecological validity.
• Findings support the method's capability to yield valuable data for decision-making research.