To be Direct or not: Reversing Likert Response Format Items

Area of Science:

• Psychometrics
• Social Sciences
• Health Sciences

Background:

• Likert items are widely used in social and health sciences.
• Acquiescence bias is a known issue with Likert scale response formats.
• Reversed items are controversially used to mitigate acquiescence bias.

Purpose of the Study:

• To investigate the impact of reversed items on the psychometric properties of a scale.
• To compare the performance of Likert scales with all positive, all negative, and mixed item formats.

Main Methods:

• Administered different versions of the Grit-s scale to an adult sample (N = 1,419).
• Analyzed psychometric properties including item analysis, dimensionality, and reliability for each scale version.
• Compared scales with all positive, all negative, and mixed positive/negative item formats.

Main Results:

• Scales with consistently positive or negative items demonstrated superior psychometric functioning compared to mixed versions.
• Despite differences in functioning, mean total scores did not significantly vary across versions.
• Mixed scales can obscure critical psychometric variations, leading to potentially misleading interpretations.

Conclusions:

• Advise against the use of mixed Likert scales due to potential masking of significant psychometric differences.
• Recommend considering exclusively positive or exclusively negative item formats for improved scale performance and validity.
• Highlight the importance of examining psychometric properties beyond mean scores when evaluating Likert scale data.