Applying Attention-Based Models for Detecting Cognitive Processes and Mental Health Conditions

Area of Science:

• Psychology
• Computational Linguistics
• Machine Learning

Background:

• Implicit motives are stable personality characteristics influencing behavior, success, and development.
• The Operant Motive Test (OMT) measures unconscious intrinsic desires through free-text responses to images and questions.
• Accurate classification of OMT responses is crucial for understanding implicit motives.

Purpose of the Study:

• To explore and compare recent machine learning techniques for the OMT classification task.
• To evaluate the performance of advanced language representations (BERT, XLM, DistilBERT) and deep supervised autoencoders.
• To analyze the contribution of writing style versus content in OMT classification.

Main Methods:

• Utilized transformer-based language models (BERT, XLM, DistilBERT) and deep supervised autoencoders.
• Compared these advanced methods against traditional classifiers like fully connected neural networks and support vector classifiers.
• Analyzed the BERT attention mechanism to understand feature importance.

Main Results:

• BERT achieved a 7.9% relative improvement over traditional machine learning techniques and the GermEval 2020 baseline.
• Transformer-based methods demonstrated superior empirical results in OMT classification.
• Features related to writing style were found to be more important than content-based words for accurate classification.

Conclusions:

• Transformer-based architectures, particularly BERT, are highly effective for OMT classification.
• Deep supervised autoencoders represent a novel approach for OMT task classification.
• Writing style analysis offers significant insights into implicit psychometrics, aligning with behavioral research.