From Likert Scales to Large Language Models: Validating a Computational Approach to Psychological Assessment of...

Area of Science:

• Psychological assessment
• Artificial intelligence in behavioral science
• Natural Language Processing (NLP)

Background:

• Traditional Likert scales have limitations in measuring complex psychological constructs.
• Large Language Models (LLMs) present novel opportunities for advanced assessment methods.
• Future Self-Continuity (FSC) is a key construct related to perceived connections between present and future selves.

Purpose of the Study:

• To introduce and validate a new LLM-based methodology for psychological assessment.
• To apply this novel NLP approach to measure Future Self-Continuity (FSC).
• To explore the clinical utility of LLM-based FSC assessment, particularly in relation to suicide ideation.

Main Methods:

• An LLM (Claude 3.5 Sonnet) was used for NLP on interview transcripts from 164 participants.
• Quantitative NLP-FSC scores were derived and compared with the Future Self-Continuity Questionnaire (FSCQ).
• LLM robustness was confirmed through replication with different model versions; clinical utility was explored using the Suicidal Behaviors Questionnaire-Revised (SBQR).

Main Results:

• NLP-FSC scores demonstrated significant convergent validity with the FSCQ (r = 0.57) and acceptable agreement via Bland-Altman analysis.
• Replication studies confirmed the robustness of the LLM-based method across different model versions (inter-model r = 0.84–0.91).
• The NLP assessment captured unique variance in suicide attempt likelihood beyond the FSCQ, indicating potential clinical relevance.

Conclusions:

• The validated LLM-based NLP approach offers a nuanced and robust method for assessing Future Self-Continuity (FSC).
• This methodology advances psychological measurement, with potential applications in both research and clinical settings.
• The approach shows promise for identifying individuals at risk, particularly concerning suicide ideation.