To Reverse Item Orientation or Not to Reverse Item Orientation, That Is the Question.

Area of Science:

• Psychological Measurement
• Social Psychology

Background:

• Psychological scales often use positively worded items.
• The impact of negatively worded items on scale properties and data interpretation is not fully understood.

Purpose of the Study:

• To investigate the psychometric properties and data interpretation effects of including negatively oriented items in established scales.
• To examine how negatively oriented items influence scale dimensionality, item functioning, and associations with other psychological variables.

Main Methods:

• Created four experimental conditions using semantic reversals of items from the Rosenberg Self Esteem Scale, UCLA Loneliness Scale, and General Belongingness Scale.
• Recruited 2,019 participants via Amazon's Mechanical Turk.
• Assessed data for dimensionality, item functioning, instrument properties, and associations with other variables using statistical modeling.

Main Results:

• A two-factor model (positive vs. negative items) showed better fit than a unidimensional model, though unidimensional interpretation was still arguable.
• Factor loadings were invariant, but thresholds differed across conditions.
• Inclusion of negative items lowered mean scores, increased score variances, and reduced reliability estimates.
• Path coefficients in mediation models varied significantly across conditions.

Conclusions:

• Negatively oriented items have a minor impact on overall instrument quality but notably influence measurement models and path coefficients.
• Findings suggest careful consideration is needed when using mixed-orientation scales to avoid misinterpretation of psychological constructs and their relationships.