From Likert to Forced Choice: Statement Parameter Invariance and Context Effects in Personality Assessment

Area of Science:

• Psychometrics
• Personality Assessment
• Psychological Measurement

Background:

• Forced-choice questionnaires (FCQs) are increasingly used for personality assessment.
• A key assumption in using FCQs is statement parameter invariance across different item formats and contexts.
• This invariance is crucial for accurate interpretation of personality measures.

Purpose of the Study:

• To empirically investigate statement parameter invariance in FCQs.
• To examine invariance between Likert and FCQ formats.
• To assess invariance between different FCQ item pairings.

Main Methods:

• An empirical study compared statement parameters using Likert and FCQ formats.
• Parameters were analyzed for common statements across two Likert and two FCQ forms.
• FCQ forms were created by pairing statements, including novel pairings and pairings from original triplet blocks.

Main Results:

• Statement parameter invariance was not supported between Likert and FCQ items.
• Invariance was also not supported for FC items when statements were recombined into new pairs.
• However, parameter invariance generally held for FC items when a statement was dropped from an original triplet block.

Conclusions:

• The assumption of statement parameter invariance in FCQs is questionable, particularly across different formats and novel item constructions.
• Contextual effects can influence item parameters, impacting the reliability of personality assessments.
• Test practitioners should exercise caution regarding parameter invariance when using FCQs and consider potential context effects.