Effects of the testing situation on item responding: cause for concern

Area of Science:

• Psychological assessment
• Personality psychology
• Measurement in psychology

Background:

• Previous research on faking in personality tests used experimental groups, impression management scores, or applicant/non-applicant comparisons.
• The Sixteen Personality Factor Questionnaire (16PF) is a widely used personality assessment tool.

Purpose of the Study:

• To investigate the effects of faking on the item and scale functioning of the Sixteen Personality Factor Questionnaire (16PF).
• To evaluate the validity of using impression management scores to identify faking in personality assessments.

Main Methods:

• Utilized job applicants and non-applicants as distinct groups to simulate faking and honest responding.
• Employed item response theory (IRT) methods, specifically detecting differential item and test functioning (DIF/DTF), to identify faking.
• Analyzed the Sixteen Personality Factor Questionnaire (16PF) data.

Main Results:

• Differential item and test functioning (DIF/DTF) was detected across different testing situations, indicating faking.
• The presence of DIF/DTF suggests that faking negatively impacts the construct validity of personality scales.
• Comparing groups defined by impression management scores proved problematic for accurately studying faking.

Conclusions:

• Faking poses a significant threat to the construct validity of personality scales like the 16PF.
• The methodology of comparing groups based on impression management scores is not recommended for studying the effects of faking.
• Further research is needed to refine methods for detecting and mitigating faking in personality assessments.