On the limitations of fixed-step-size adaptive methods with response confidence

Area of Science:

• Psychophysics
• Statistical methods

Background:

• Adaptive methods, including up-down and staircase procedures, are widely used in psychophysical studies for estimating sensory thresholds.
• Extensions of these methods to handle non-binary responses, such as the weighted up-down (WUD) method, have been proposed but possess limitations.

Purpose of the Study:

• To analyze the limitations of existing extensions of adaptive methods for non-binary responses.
• To introduce and evaluate an algorithm that incorporates response confidence into adaptive threshold estimation methods.
• To compare the performance of the proposed algorithm with existing methods like Derman's up-down and the biased-coin design.

Main Methods:

• Characterization of the weighted up-down (WUD) method extension using an algorithm that integrates response confidence.
• Application of this algorithm to Derman's up-down method and the biased-coin design for threshold quantile estimation.
• Simulation studies to assess the limitations of the algorithm across the three adaptive methods.
• Experimental evaluation of the extended WUD with varying response confidence formats to assess threshold estimation consistency.

Main Results:

• The study identifies and discusses inherent limitations in extending adaptive methods, specifically WUD, to accommodate non-binary response categories.
• Simulations reveal performance constraints of the proposed confidence-incorporating algorithm when applied to WUD, Derman's up-down, and biased-coin designs.
• Experimental results using the extended WUD demonstrate the impact of different response confidence formats on the consistency of threshold estimation.

Conclusions:

• Existing extensions of adaptive methods for non-binary responses, including WUD, face significant limitations.
• An algorithm incorporating response confidence offers a framework for extending adaptive methods but requires careful consideration of its limitations.
• Further research and methodological refinement are needed to improve the accuracy and consistency of adaptive threshold estimation with non-binary responses and confidence ratings.