Verbal lie detection using Large Language Models.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Artificial Intelligence

Background:

• Human lie detection accuracy is limited, often not exceeding chance levels.
• Automated verbal lie detection methods using Machine Learning and Transformer models have been developed to improve accuracy.
• Large Language Models (LLMs) represent a novel approach for enhancing deception detection capabilities.

Purpose of the Study:

• To evaluate the performance of FLAN-T5, a Large Language Model, in classifying deception across diverse English-language datasets.
• To compare the effectiveness of small and base FLAN-T5 model sizes for lie detection.
• To investigate the impact of dataset composition and model size on deception detection accuracy.

Main Methods:

• Stylometric analysis was conducted to identify linguistic differences across datasets (personal opinions, autobiographical memories, future intentions).
• FLAN-T5 (small and base sizes) was tested in three cross-validation scenarios, varying data distribution between training and testing sets.
• Performance was evaluated using 10-fold cross-validation, comparing results against existing benchmarks.

Main Results:

• FLAN-T5 achieved state-of-the-art performance in Scenarios 1 and 3, surpassing previous benchmarks.
• Model performance was positively correlated with model size; larger FLAN-T5 models demonstrated higher accuracy.
• Stylometric analysis indicated that linguistic features linked to the Cognitive Load framework may significantly influence model predictions.

Conclusions:

• FLAN-T5 demonstrates significant potential as an automated tool for verbal lie detection.
• Model size is a critical factor in achieving optimal performance for deception detection tasks.
• Understanding linguistic features, particularly those related to cognitive load, can enhance the explainability and effectiveness of LLM-based lie detection systems.