Lying on the Dissection Table: Anatomizing Faked Responses

Area of Science:

• Psychological Measurement
• Machine Learning Applications

Background:

• Detecting response faking in research is challenging, with experts performing at chance levels.
• Previous machine learning efforts overlooked variations in faking conditions and the integration of faking indices.

Purpose of the Study:

• To investigate machine learning's effectiveness in detecting faking across diverse conditions.
• To compare different input data types and machine learning classifiers for faking detection.
• To identify features utilized by classifiers in detecting faking.

Main Methods:

• Reanalyzed seven datasets (N=1,039) encompassing various faking conditions (e.g., high/low scores, self-reports vs. Implicit Association Tests [IATs]).
• Compared logistic regression, random forest, and XGBoost classifiers using response patterns, scores, and faking indices as input.
• Explored classifier feature importance for faking detection.

Main Results:

• Machine learning detection success varied significantly, from chance to 100%, depending on the faking condition.
• Low-score faking was more detectable than high-score faking.
• For self-reports, response patterns and scores were comparable; for IATs, faking indices and response patterns outperformed scores.
• Logistic regression and random forest performed similarly and better than XGBoost.
• Classifiers often used multiple features, indicating complex faking processes.

Conclusions:

• Machine learning offers a promising avenue for detecting response faking, but its efficacy is condition-dependent.
• Acknowledging diverse faking processes and incorporating faking indices are crucial for advancing detection methods.
• Understanding feature relevance aids in interpreting classifier performance and the nature of faking.