Story Arcs in Serious Illness: Natural Language Processing features of Palliative Care Conversations

Serious illness conversations are complex clinical narratives that remain poorly understood. Natural Language Processing (NLP) offers new approaches for identifying hidden patterns within the lexicon of stories that may reveal insights about the taxonomy of serious illness conversations.

We analyzed verbatim transcripts from 354 consultations involving 231 patients and 45 palliative care clinicians from the Palliative Care Communication Research Initiative. We stratified each conversation into deciles of "narrative time" based on word counts. We used standard NLP analyses to examine the frequency and distribution of words and phrases indicating temporal reference, illness terminology, sentiment and modal verbs (indicating possibility/desirability).

Temporal references shifted steadily from talking about the past to talking about the future over deciles of narrative time. Conversations progressed incrementally from "sadder" to "happier" lexicon; reduction in illness terminology accounted substantially for this pattern. We observed the following sequence in peak frequency over narrative time: symptom terms, treatment terms, prognosis terms and modal verbs indicating possibility.

NLP methods can identify narrative arcs in serious illness conversations.

Fully automating NLP methods will allow for efficient, large scale and real time measurement of serious illness conversations for research, education and system re-design.