Beyond Scales and Essays: Artificial Intelligence Chatbots as Personality Interviewers

Area of Science:

• Psychology
• Artificial Intelligence
• Human-Computer Interaction

Background:

• AI chatbots provide interactive personality assessment via semi-structured interviews.
• Leveraging NLP and ML, chatbots offer nuanced, contextual personality evaluations.
• This study compares AI chatbot validity against traditional personality assessment methods.

Purpose of the Study:

• To compare the psychometric validity of AI chatbot personality interviews with conventional methods.
• To examine the influence of user perceptions on AI chatbot personality assessment validity.
• To explore the incremental validity of AI chatbot scores in predicting outcomes.

Main Methods:

• Utilized a multitrait-multimethod (MTMM) framework and generalizability theory (G-theory).
• Collected data from 189 participants at two time points.
• Compared AI chatbot interviews with scale-based measures and short essay questionnaires.

Main Results:

• Machine-inferred scores from AI chatbots and essays showed good convergence with scale-based methods but weaker discriminant validity.
• Chatbot-based scores demonstrated incremental validity for predicting subjective health and perceived job performance.
• Ease of concentration during chatbot interviews may influence score convergence.

Conclusions:

• AI chatbots show promise for personality assessment, complementing traditional methods.
• Further research is needed on user perceptions' impact on AI-based assessment psychometrics.
• Chatbot assessments offer unique predictive validity for specific outcomes.