A psychometric framework for evaluating and shaping personality traits in large language models.

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Psychometrics

Background:

• Large language models (LLMs) exhibit synthetic personality traits due to training data.
• These traits influence communication effectiveness in LLM-powered applications.
• Evaluating and shaping LLM personality is crucial for widespread public use.

Purpose of the Study:

• To present a psychometric methodology for assessing LLM personality.
• To validate personality measurement in LLM outputs.
• To enable shaping of LLM personality in generated text.

Main Methods:

• Administering and validating personality tests on 18 large language models.
• Utilizing specific prompting configurations for personality assessment.
• Developing methods to shape LLM personality along desired dimensions.

Main Results:

• Personality measurements in LLM outputs can be reliable and valid under certain conditions.
• Larger, instruction-fine-tuned models show stronger evidence of reliable and valid synthetic personality.
• LLM personality can be successfully shaped to mimic specific human profiles.

Conclusions:

• A robust psychometric methodology for LLM personality assessment and shaping has been established.
• The findings support the reliable measurement and intentional modification of synthetic personalities in LLMs.
• Ethical considerations for responsible AI development regarding LLM personality are discussed.