How humans impair automated deception detection performance.

Area of Science:

• Psychology
• Computer Science
• Security Studies

Background:

• Deception detection is a significant challenge for security professionals.
• Automated machine learning methods offer scalable solutions but have error limitations.
• Hybrid human-machine approaches are explored to enhance detection accuracy.

Purpose of the Study:

• To evaluate if combining supervised machine learning with human judgment improves deception detection accuracy.
• To assess the impact of human override and adjustment capabilities in a hybrid system.
• To investigate the influence of human decision-making biases on automated deception detection.

Main Methods:

• Collected a corpus of 1640 truthful and deceptive autobiographical intention statements.
• Tested two hybrid conditions: human judges could fully overrule or adjust machine learning credibility judgments.
• Compared accuracy of machine learning alone, hybrid-overrule, and hybrid-adjust conditions.

Main Results:

• Machine learning alone achieved 69% accuracy in identifying truth-tellers and liars.
• Neither hybrid condition improved deception detection accuracy.
• Human involvement in the hybrid-overrule condition reduced accuracy to chance level.
• The hybrid-adjust condition also failed to enhance performance, potentially due to human truth bias.

Conclusions:

• Human judgment does not meaningfully enhance the deception detection performance of machine learning systems.
• The study does not support the integration of human oversight to improve automated lie detection.
• Truth bias may explain the detrimental effect of human involvement in these hybrid systems.