Navigating pathways to automated personality prediction: a comparative study of small and medium language models

Area of Science:

• Computational linguistics
• Natural Language Processing (NLP)
• Personality prediction

Background:

• Advancements in NLP and social media data enable personality prediction.
• Large Language Models (LLMs) excel but have high computational costs (carbon/water footprint).
• Trend towards smaller, efficient models is observed.

Purpose of the Study:

• Compare the performance of a small model (ALBERT) and a large model (RoBERTa) for predicting big five personality traits.
• Investigate the impact of model size versus data quality on prediction accuracy.
• Evaluate computational resource consumption.

Main Methods:

• Utilized the PANDORA dataset (Reddit comments) for training and evaluation.
• Employed ALBERT (11.8M parameters) and RoBERTa (125M parameters).
• Customized models for multi-output regression with added linear layers for fine-grained analysis on a Tesla P100 GPU.

Main Results:

• ALBERT consumed less memory and generated less heat but required more computation time than RoBERTa.
• Both models achieved comparable Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and training loss reduction.
• Training data quality significantly influenced model performance.

Conclusions:

• Model size is less critical than training data quality for personality prediction accuracy.
• Efficient smaller models can achieve performance comparable to larger models.
• Findings have implications for resource-efficient NLP applications in personality analysis.