Lie Recognition with Multi-Modal Spatial-Temporal State Transition Patterns Based on Hybrid Convolutional Neural...

Area of Science:

• Cognitive Science
• Computer Vision
• Forensic Science

Background:

• Lie recognition is cognitively complex, with limited success in current methods due to the subtlety of involuntary cues.
• Existing approaches for lie detection in legal and investigative settings often struggle with performance and generalization.
• The scarcity of reliable involuntary features hinders the accuracy of automated lie detection systems.

Purpose of the Study:

• To enhance lie recognition performance by developing a novel approach utilizing involuntary cues.
• To introduce a multi-modal feature model integrating hand, body, and eye blinking patterns.
• To improve the accuracy and generalizability of deception detection in forensic contexts.

Main Methods:

• Extracted spatial-temporal state transition patterns (STSTP) from hand and face poses using convolutional neural networks (ResNet-152).
• Computed eye blinking features via an eye aspect ratio model.
• Fused these features into a multi-modal model enhanced with bidirectional long short-term memory (BiLSTM).

Main Results:

• The proposed multi-modal STSTP feature model demonstrated improved lie detection performance compared to state-of-the-art methods.
• Integration of involuntary cues from body motion and eye blinks proved effective in enhancing deception detection.
• The model showed superior accuracy in identifying lies within real-life court trial videos.

Conclusions:

• The developed STSTP multi-modal model offers a promising advancement in automated lie recognition.
• Leveraging involuntary cognitive cues through advanced feature extraction and fusion enhances deception detection capabilities.
• This approach has significant potential for application in legal and investigative settings to improve the reliability of truth verification.