Measuring Response Style Stability Across Constructs With Item Response Trees.

Area of Science:

• Psychometrics
• Survey Methodology
• Statistical Modeling

Background:

• Response styles can bias survey scores, impacting the valid interpretation of latent traits.
• Previous research on response style stability across scales is limited, particularly from the item response trees perspective.
• Item response trees (IRTrees) offer a framework to model response styles and latent traits simultaneously.

Purpose of the Study:

• To extend the IRTree methodology to accommodate mixed item formats.
• To investigate the stability of response styles across multiple scales within a single survey administration.
• To examine the impact of response styles on latent trait estimation.

Main Methods:

• An extended IRTree methodology was applied to analyze responses from three scales (social media, climate change, medical marijuana).
• The study incorporated mixed item formats (e.g., 4-point and 5-point Likert-type items).
• Bayesian Markov chain Monte Carlo estimation was used for trait estimation.

Main Results:

• Extreme and midpoint response styles were found to be unstable across the different scales.
• Five-point Likert-type items showed higher levels of extreme response style compared to four-point items.
• Latent trait estimation varied significantly across different response style models, especially for lower score ranges.

Conclusions:

• Response style stability is not guaranteed across scales, even within a single administration.
• The choice of response style model significantly impacts the accuracy of latent trait estimation.
• Accurate trait estimation requires careful consideration and modeling of individual response styles, particularly with mixed item formats.