Spurious Reliability Increase?: The Number of Response Options in the Likert-Type Scale Influences Only Internal...

Area of Science:

• Psychometrics
• Survey Methodology

Background:

• Existing research on Likert-type response formats primarily examines reliability, often neglecting validity.
• This study addresses the gap by investigating the impact of varying response options on both reliability and validity.

Purpose of the Study:

• To examine how the number of response options (two, six, and 10) affects the reliability and validity of Likert-type scales.
• To determine if increased reliability with more options is genuine or due to construct-irrelevant variance.

Main Methods:

• A within-subject experiment with 846 participants (69% women).
• Manipulation of response options (two, six, 10) in the Height Inventory and the autonomy subscale of the Basic Needs Satisfaction in General Scale.
• Analysis of means, reliability, and various validity measures (criterion, measurement model, trait criterion validity).

Main Results:

• Two-point response scales significantly differed in means and reliability compared to six- and 10-point scales.
• Validity measures remained unaffected across different response formats.
• Observed reliability increases with more options may be spurious, potentially due to method variance.

Conclusions:

• Response formats with fewer options, particularly two-point scales, can be viable for Likert-type measures, especially when scales have more items.
• The choice of response format may influence reliability through construct-irrelevant variance rather than enhancing true score measurement.
• Future research should investigate cognitive processes underlying responses across different format options.