Evaluating the ability of large language models to emulate personality.

Area of Science:

• Social Sciences
• Artificial Intelligence
• Psychology

Background:

• Large Language Models (LLMs) offer potential for simulating human behavior in social sciences.
• Realistic agent creation with diverse individual differences is crucial for behavioral studies.

Purpose of the Study:

• To assess GPT-4's capability in role-playing individuals with diverse Big Five personality profiles.
• To evaluate the validity and consistency of LLM-emulated personality responses.

Main Methods:

• Simulation studies using GPT-4 to emulate real-world individuals.
• Analysis of internal consistency, factor organization, and convergent validity of emulated personality scores.
• Assessment of LLM performance with varying role complexity and inclusion of demographic data.

Main Results:

• Emulated personality responses showed high internal consistency and distinct factor organization.
• Emulated scores demonstrated strong convergent validity with human self-reported personality.
• Role-playing robustness decreased with increased role complexity; demographic data improved external criterion prediction.

Conclusions:

• LLMs show promise for creating realistic, personality-diverse agents for social science research.
• Further research is needed to address limitations related to role complexity and data integration.
• LLM-emulated agents can be valuable tools for understanding human behavior and individual differences.