Comparison of methods for quick estimation of psychometric thresholds

Area of Science:

• Cognitive Science
• Psychophysics
• Computational Neuroscience

Background:

• Threshold estimation is fundamental in cognitive science for understanding perception.
• Existing psychophysical methods aim for rapid estimation of psychometric function parameters, such as thresholds.
• Classical methods often assume monotonic psychometric functions, but real-world data can exhibit non-monotonicity.

Purpose of the Study:

• To compare the efficacy of various stimulus selection methods for threshold estimation using Monte Carlo simulations.
• To evaluate model performance based on parameter bias, interquartile range, limits of agreement, and miss rates for random responders.
• To investigate methods capable of estimating thresholds for both monotonic and non-monotonic psychometric functions, and identify random or outlier participants.

Main Methods:

• Monte Carlo simulations with 60 to 120 trials were employed to assess threshold estimation methods.
• Performance was evaluated using metrics like parameter bias, interquartile range, limits of agreement, and miss rate.
• Methods were tested under conditions of monotonic and non-monotonic psychometric functions, including random responding and outlier parameters.

Main Results:

• Bayesian methods demonstrated an order-of-magnitude improvement over the Method Of Constant Stimuli (MOCS) for monotonic functions.
• Assuming monotonicity for non-monotonic functions led to extreme biases (over 4,000%) even with 120 trials.
• The psi-marginal and psi-marg-grid methods showed superior performance across all scenarios, accurately estimating thresholds for both monotonic and non-monotonic functions.
• MOCS, Psi, and psi-grid struggled to identify random responders (miss rates > 21%).
• The recommended psi-marginal and psi-marg-grid methods exhibited minimal performance cost when assuming non-monotonicity for truly monotonic functions.

Conclusions:

• The psi-marginal and psi-marg-grid methods are highly recommended for efficient and robust threshold estimation in cognitive science.
• These methods provide accurate results regardless of whether psychometric functions are monotonic or non-monotonic.
• Adopting these Bayesian approaches enhances the reliability of threshold parameter estimation, even with limited trials or unusual participant responses.