Leveraging Large Language Models for Improved Understanding of Communications With Patients With Cancer in a Call...

Area of Science:

• Artificial Intelligence in Oncology
• Natural Language Processing for Healthcare

Background:

• Hospital call centers are vital for cancer patient support, necessitating accurate intent identification.
• Traditional deep learning models (LSTM, BERT) require extensive annotated data, posing efficiency challenges.
• Large language models (LLMs) offer a promising alternative for intent classification with in-context learning.

Purpose of the Study:

• To evaluate GPT-4's performance in classifying patient intent during cancer-related telephone consultations.
• To compare GPT-4's accuracy against LSTM and BERT, particularly for ambiguous and complex queries.

Main Methods:

• Utilized a dataset of 430,355 sentences from cancer patient telephone consultations (2016-2020).
• Trained LSTM and BERT models using supervised learning on 300,000 sentences.
• Applied GPT-4 using zero-shot and few-shot approaches without retraining, comparing accuracy on 1,000 sentences.

Main Results:

• GPT-4 achieved 85.2% accuracy, significantly outperforming LSTM (73.7%) and BERT (71.3%) in few-shot learning.
• GPT-4 showed over 15% improvement in handling ambiguous queries within complex categories like 'Treatment' and 'Symptoms'.
• GPT-4 excelled in clear-context categories ('Records,' 'Routine'), demonstrating its versatility in interpreting patient interactions.

Conclusions:

• GPT-4 shows significant potential to enhance patient intent classification in oncological consultations.
• Its ability to manage complex queries without retraining offers a key advantage over existing discriminative models.
• Further research on prompt design and hybrid AI systems is recommended for practical healthcare integration.