Reasoning beyond clicks: Disentangling counterexamples and probabilities in conditional reasoning.

Area of Science:

• Cognitive Psychology
• Human Reasoning
• Decision Making

Background:

• Traditional reasoning experiments use fixed response formats (e.g., scales, multiple-choice).
• These formats may inadvertently influence cognitive reasoning strategies and introduce bias.
• An open response format allows for more naturalistic expression of conclusions and reasoning processes.

Purpose of the Study:

• To investigate human reasoning strategies using an open response format.
• To identify and categorize the types of conclusions and justifications participants provide.
• To introduce and define the concept of 'probabilistic masking'.

Main Methods:

• Two experiments were conducted involving participants generating written conclusions.
• Over 1,300 free-text responses were collected and analyzed.
• A predefined coding scheme categorized responses as certain, uncertain (counterexamples), or uncertain (probabilities).

Main Results:

• Participants frequently provided certain conclusions rather than expressing nuanced beliefs.
• When uncertainty was expressed, it was more often supported by concrete counterexamples than probabilistic reasoning.
• The study observed a tendency for certain responses over graded beliefs.

Conclusions:

• Open response formats reveal different reasoning patterns compared to traditional methods.
• Probabilistic masking, where graded response formats bias towards probabilistic interpretations, was identified.
• Findings suggest current methods may overestimate the role of probabilistic reasoning in human inference.