Can large language models code meaning-making? A multi-model validation of automated autobiographical reasoning...

Autobiographical reasoning (AR) involves drawing self-relevant meaning from personal experiences. It is central to narrative identity research, but manual coding requires substantial time and trained coders. Large language models (LLMs) have shown promise for automating other narrative coding tasks, but their capacity for meaning-making judgments that may depend on implicit event-self connections remains untested. We collected autobiographical narratives from 437 Chinese high school students and evaluated three LLMs (GPT-4o, DeepSeek-V3, Qwen-Max) across four prompting strategies (zero-shot, chain-of-thought, 3-shot, 5-shot) using three scoring granularities (binary, ordinal, continuous). The best-performing condition (DeepSeek-V3, zero-shot) achieved Run 1 Cohen's κ = .416; three-run majority-vote accuracy rose from 54.9% for AR-absent texts to 94.4% for moderate-to-high-AR texts, revealing strongly condition-dependent performance. In the fully crossed subset, paired-bootstrap confidence intervals for CoT-minus-zero-shot differences included zero for all three models. Error analysis identified two bias patterns: false negatives concentrated in low-AR texts (OR = 3.99, p < .001) and false positives consistent with over-interpretation of emotional expression. Under the models, prompts, corpus, and scoring rules tested here, general-purpose LLMs cannot replace formal manual AR coding.