Automated scoring of the autobiographical interview with natural language processing

Area of Science:

• Psychology
• Cognitive Science
• Computational Linguistics

Background:

• The autobiographical interview is a key method for assessing autobiographical memory content.
• Manual scoring of these memories is time-consuming and limits study size.
• Automated scoring is needed to reduce burden and facilitate larger research endeavors.

Purpose of the Study:

• To develop and validate a natural language processing (NLP) approach for automatically scoring autobiographical narratives.
• To quantify internal (episodic) and external (non-episodic) details within memories.
• To enable larger and more efficient studies on autobiographical memory.

Main Methods:

• Fine-tuning a distilBERT language model to identify internal and external content at the sentence level.
• Aggregating sentence-level predictions to estimate content for entire narratives.
• Evaluating the automated model against manual scores across five distinct datasets.

Main Results:

• The NLP model demonstrated strong performance across multiple datasets.
• High correlations were observed between manual and automated scores for both internal and external details in four datasets.
• Minimal misclassification of content was found, with good performance on a fifth dataset after preprocessing.

Conclusions:

• Automated scoring using fine-tuned NLP models is a viable and efficient alternative to manual scoring of autobiographical memories.
• This approach significantly reduces scoring time and burden, paving the way for larger-scale investigations.
• A provided Colab notebook makes this automated scoring tool accessible to researchers without requiring additional coding expertise.