Traditional difference-score analyses of reasoning are flawed.

Area of Science:

• Cognitive Psychology
• Human Reasoning

Background:

• Belief bias effect in syllogistic reasoning is traditionally measured using logic, belief, and interaction indices.
• Concerns exist that the interaction index incorrectly assumes linear receiver operating characteristic (ROC) curves.

Purpose of the Study:

• To evaluate traditional measures of belief bias in syllogistic reasoning.
• To investigate the impact of instructional manipulations on belief bias.
• To compare traditional measures with signal detection theory (SDT) analyses.

Main Methods:

• Simulations of reasoning experiments using traditional difference score measures.
• Two new experiments examining instructional manipulations on belief bias.
• Signal detection theory (SDT) model-based analyses of ROC curves.
• Reanalysis of previous conditional reasoning studies.

Main Results:

• Simulations indicated traditional analyses risk incorrect conclusions due to non-linear ROCs.
• Experimental data showed ROCs violated assumptions of traditional measures.
• SDT analyses provided a better fit to ROCs, indicating belief bias is mainly a response bias.
• Reanalyses confirmed non-linear ROCs in conditional reasoning.

Conclusions:

• Traditional measures in reasoning research are unreliable due to violated assumptions.
• Signal detection theory (SDT) offers a more accurate framework for analyzing belief bias.
• Belief bias and instructional effects appear to be predominantly response bias phenomena.