Automated question type coding of forensic interviews and trial testimony in child sexual abuse cases

Question-type classification is widely used as a measure of interview quality. However, question-type coding is a time-consuming process when performed by manual coders. Reliable automated question-type coding approaches would facilitate the assessment of the quality of forensic interviews and court testimony involving victims of child abuse.

We expected that the reliability achieved by the automated model would be comparable to manual coders.

We examined whether a large language model (Robustly Optimized Bidirectional Encoder Representations from Transformers Approach) trained on questions (N = 351,920) asked in forensic interviews (n = 1,435) and trial testimony (n = 416) involving 3- to 17-year-old alleged victims of child sexual abuse could distinguish among (a) invitations, (b) wh-questions, (c) option-posing questions, and (d) nonquestions.

The model achieved high reliability (95% agreement; κ = .93). To determine whether disagreements were due to machine or manual errors, we recoded inconsistencies between the machine and manual codes. Manual coders erred more often than the machine, particularly by overlooking invitations and nonquestions. Correcting errors in the manual codes further increased the model's reliability (98% agreement; κ = .97).

Automated question-type coding can provide a time-efficient and highly accurate alternative to manual coding. We have made the trained model publicly available for use by researchers and practitioners. (PsycInfo Database Record (c) 2025 APA, all rights reserved).