How experimental procedures influence estimates of metacognitive ability

Area of Science:

• Cognitive psychology
• Neuroscience
• Psychophysics

Background:

• Accurate metacognitive ability assessment requires equating stimulus sensitivity.
• Continuous staircase procedures are commonly used to equate stimulus sensitivity during experiments.
• The assumption that staircase procedures do not influence metacognitive ability estimates is widely accepted but critically examined here.

Purpose of the Study:

• To critically examine the assumption that continuous staircase procedures do not influence metacognitive ability estimates.
• To investigate whether staircase techniques inflate metacognitive sensitivity compared to single-level stimulus contrast.
• To identify factors correlating with metacognitive inflation in staircase designs.

Main Methods:

• Analysis of previously published data involving metacognition experiments.
• Comparison of metacognitive sensitivity estimates obtained using continuous staircase versus single-level stimulus contrast.
• Examination of various metacognitive measures, including area under the type 2 ROC curve, confidence-accuracy correlation (phi), meta-d', meta-d'/d', and meta-d'-d'.

Main Results:

• Staircase techniques significantly inflate estimates of metacognitive ability across multiple measures compared to single-level contrast.
• Metacognitive inflation is positively correlated with the degree of stimulus variability experienced by subjects.
• The findings suggest that staircase procedures likely lead to overestimations of metacognitive ability.

Conclusions:

• Studies employing staircase procedures may report inflated metacognitive ability estimates.
• Variability in task difficulty, from factors like alertness or learning, can also cause similar inflation.
• Practical solutions for designing and analyzing metacognition experiments to mitigate these biases are proposed.