Identifying single-item faked responses in personality tests: A new TF-IDF-based method

Area of Science:

• Psychological assessment
• Computational linguistics
• Information retrieval

Background:

• Faking responses in psychological tests is common when examinees perceive personal gain.
• Current detection methods identify fakers but not specific faked items.
• Need for precise tools to identify individual distorted responses in psychological assessments.

Purpose of the Study:

• To evaluate the effectiveness of term frequency-inverse document frequency (TF-IDF) in identifying single-item faked responses.
• To assess TF-IDF's capability in distinguishing honest from dishonest answers within psychological questionnaires.
• To pinpoint specific questions distorted by faking behavior.

Main Methods:

• Utilized term frequency-inverse document frequency (TF-IDF), a mathematical tool from information retrieval.
• Applied the TF-IDF model to three datasets of responses to the 10-item Big Five questionnaire.
• Validated the technique across three distinct faking scenarios (child custody, job applications) with 694 participants.

Main Results:

• The TF-IDF model demonstrated high effectiveness in differentiating honest from dishonest responses.
• Honest responses consistently yielded lower TF-IDF values, while faked responses showed higher values.
• The technique successfully identified which specific items within the 10-item questionnaire were distorted due to faking.

Conclusions:

• TF-IDF is a powerful tool for detecting faked responses in psychological testing at the item level.
• This method enhances the accuracy of psychological assessments by identifying specific instances of response distortion.
• The study provides a novel, computationally-driven approach to analyzing response validity in psychometric evaluations.