Assessing personality using zero-shot generative AI scoring of brief open-ended text

Area of Science:

• Psychology
• Artificial Intelligence
• Computational Social Science

Background:

• Traditional personality assessment uses psychometric scales, which may oversimplify personality.
• Generative large language models (LLMs) offer a novel approach to analyzing qualitative data.

Purpose of the Study:

• To explore the use of commercial generative LLMs for personality trait assessment from qualitative narratives.
• To evaluate the convergence and predictive validity of LLM-based personality scores.

Main Methods:

• Utilized seven commercial generative LLMs to score Big-Five personality traits from spontaneous thought streams and daily video diaries.
• Compared LLM-derived scores with self-report measures and established benchmarks.
• Assessed the predictive validity of LLM-generated scores on daily behaviors and mental health outcomes.

Main Results:

• LLM-based personality assessments showed convergence with self-report measures, comparable to or exceeding existing benchmarks.
• Averaging scores across multiple LLMs yielded the strongest agreement with self-report.
• LLM-generated trait scores predicted daily behaviors and mental health outcomes.

Conclusions:

• Generative LLMs can provide quantitative rigor for personality assessment using qualitative data.
• This LLM-based method is accessible without specialized training and highlights the value of idiographic narratives.
• Findings encourage the adoption of LLMs for psychological assessment and underscore the ubiquity of personality expression.