A comparison of the response-pattern-based faking detection methods.

Area of Science:

• Psychological assessment
• Personality psychology
• Psychometrics

Background:

• Response-pattern-based (RPB) methods are key for detecting faking on personality tests.
• Limited understanding exists regarding how factors like scale length and sample size impact RPB method performance.
• Covariance index, idiosyncratic item response, and machine learning are primary RPB approaches.

Purpose of the Study:

• To systematically compare the performance of three RPB faking detection methods.
• To investigate the influence of practical factors on RPB method efficacy.
• To provide guidance for optimal implementation of faking detection techniques.

Main Methods:

• Conducted three resampling studies using empirical data.
• Systematically varied conditions such as scale length, training sample size, and proportion of fakers.
• Compared covariance index, idiosyncratic item response, and machine learning methods.

Main Results:

• Machine learning method demonstrated superior performance across most simulated conditions.
• All RPB methods showed moderate to strong positive correlations between faking probabilities and true personality scores.
• Potential for misclassification of honest respondents with high trait scores was identified.

Conclusions:

• Machine learning is the most effective RPB method for faking detection.
• Despite potential misclassification, the benefits of identifying fakers outweigh the risks.
• Practical guidance and code are provided for implementing the machine learning method.