A critical analysis of MBTI-based personality profiling with large language models

Area of Science:

• Artificial Intelligence
• Psychology
• Computational Linguistics

Background:

• The Myers-Briggs Type Indicator (MBTI) is widely used for personality profiling.
• Large Language Models (LLMs) are increasingly explored for their potential in understanding and simulating human cognition, including personality.

Purpose of the Study:

• To critically analyze the efficacy and limitations of using LLMs for MBTI-based personality profiling.
• To evaluate LLMs both as tools for inferring human personality and as subjects assessed by psychometric frameworks.

Main Methods:

• Review of recent research (2020-2025) on LLM applications in personality assessment.
• Analysis of various LLM approaches including traditional machine learning, fine-tuned transformers, and zero-shot prompting.
• Examination of performance across diverse datasets (Kaggle MBTI, PersonalityCafe, Pandora, MBTIBench) and evaluation using soft labels.

Main Results:

• Top LLM systems achieve 75%-85% accuracy at the dichotomy level, but improvements over baselines are often modest and dataset-dependent.
• LLMs exhibit systematic issues like polarized predictions, overconfidence, and poor calibration.
• LLMs demonstrate reproducible but context-dependent 'personality-like' profiles, often skewed towards socially desirable traits due to alignment training.

Conclusions:

• MBTI-based LLM personality modeling is hindered by the MBTI's psychometric limitations, weaknesses in self-reported data, and the philosophical ambiguity of AI personality.
• Ethical risks and evaluation gaps necessitate more rigorous, calibrated, and theoretically grounded approaches to AI personality modeling.