Systemic Anticancer Therapy Timelines Extraction From Electronic Medical Records Text: Algorithm Development and...

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Bioinformatics

Background:

• Systemic anticancer therapy (SACT) often involves complex drug combinations and sequences.
• Clinical narratives in electronic medical records (EMRs) contain detailed SACT timelines.
• Automated extraction of these timelines is a significant challenge.

Purpose of the Study:

• To explore automatic methods for extracting patient-level SACT timelines from clinical narratives in EMRs.
• To compare the performance of finetuned language models and large language models (LLMs) for this task.

Main Methods:

• Utilized two datasets: THYME (colorectal cancer) and ChemoTimelines shared task (ovarian, breast cancer, melanoma).
• Explored finetuning smaller language models (EntityBERT) and few-shot prompting of LLMs (LLaMA, Mixtral).
• Evaluated performance on Subtask1 (timeline construction from annotated input) and Subtask2 (direct extraction from notes).

Main Results:

• The finetuned EntityBERT model achieved a 93% F1-score, surpassing the best Subtask1 result (90%) in the ChemoTimelines shared task.
• EntityBERT ranked second in Subtask2.
• LLMs (LLaMA2, LLaMA3.1, Mixtral) underperformed the finetuned model, with the best LLM achieving 77% macro F1-score on shared task datasets (Subtask1).

Conclusions:

• Task-specific finetuning of language models, like EntityBERT, is highly effective for extracting SACT timelines from clinical narratives.
• This approach outperforms general-purpose LLMs for this specialized task.
• The findings contribute to advancing automated treatment timeline extraction from EMRs.