Multimodal Behavioral Sensors for Lie Detection: Integrating Visual, Auditory, and Generative Reasoning Cues

Area of Science:

• Artificial Intelligence
• Behavioral Science
• Computational Linguistics

Background:

• Multimodal AI advances enable novel sensor-inspired lie detection.
• Integrating behavioral perception with generative reasoning is key.

Purpose of the Study:

• To present a deception detection framework using deep video/audio processing and large language models.
• To enable interpretable deception hypotheses generation via chain-of-thought (CoT) prompting and prompt-level fusion.

Main Methods:

• Utilized ViViT and HuBERT as digital behavioral sensors for extracting emotional and cognitive cues.
• Employed GPT-5 for prompt-level fusion, aligning video, language, and emotion for zero-shot inference.
• Processed visual frames, transcripts, and emotion recognition outputs on the DOLOS dataset.

Main Results:

• Multimodal fusion and CoT reasoning significantly improved deception classification accuracy.
• The system demonstrated interpretable deception hypothesis generation without task-specific fine-tuning.
• Continual learning setup showed effective transfer of emotional understanding to deception classification.

Conclusions:

• The proposed AI framework effectively bridges raw behavioral data and semantic inference for lie detection.
• This approach lays the foundation for AI-driven deception detection with interpretable sensor analogues.
• Sensor-inspired AI offers a promising direction for understanding and detecting deceptive behaviors.