Dichotomous versus polytomous response options in psychopathology assessment: method or meaningful variance?

In previous studies, researchers have examined the optimal number of response options for psychological questionnaires. Several have reported increased scale score reliabilities, but few have documented improved external validities. In the present investigation, we followed-up on Cox (2011) and Cox et al.'s (2012) extensive analyses of a clinical assessment instrument, the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF). We compared the dichotomous (true/false) response format of this inventory with a 4-choice format. Our sample consisted of 406 undergraduate students from a large Midwestern university who were largely female (64.3%), predominantly Caucasian (76.4%), and had a mean age of 19.24 years. Internal-structural analyses confirmed that more response options increase reliabilities, but the effects were small. The differences between correlations with external criteria were very rarely statistically significant, and the few that were did not consistently favor either format. We recommend that in future response-format investigations the internal-structural analyses continue to be combined with evaluations of relevant external correlations.