Development of a Computerized Adaptive Item Bank for the Big Five Personality Based on Large Language Models.

Area of Science:

• Psychometrics
• Artificial Intelligence
• Natural Language Processing

Background:

• Traditional item development limits computerized adaptive testing (CAT) and intelligent assessment.
• Automatic item generation (AIG) using natural language processing (NLP) offers a solution.

Purpose of the Study:

• To explore the use of large language models (LLMs) for generating Simplified Chinese Big Five personality items.
• To evaluate the effectiveness of LLM-generated item banks for CAT.

Main Methods:

• Three LLMs were utilized to generate personality assessment items.
• Two rounds of empirical testing were conducted to validate the item bank's performance.

Main Results:

• LLM-based AIG proved to be a cost-effective and efficient method for creating high-quality Big Five CAT item banks.
• The approach demonstrated consistent performance across different LLMs, indicating cross-model stability.

Conclusions:

• LLM-based AIG is a viable strategy for developing intelligent assessment tools.
• This technology addresses key challenges in CAT development, paving the way for automated assessment workflows.