Developing a named entity framework for thyroid cancer staging and risk level classification using large language...

Area of Science:

• Oncology
• Medical Informatics
• Artificial Intelligence

Background:

• Classifying well-differentiated thyroid cancer staging and risk is crucial for patient management.
• Extracting this information from clinical notes can be challenging due to semi-structured data.
• Existing methods may lack efficiency and accuracy in processing large datasets.

Purpose of the Study:

• To develop a named entity (NE) framework for information extraction from The Cancer Genome Atlas-Thyroid Cancer (TCGA-THCA) database.
• To evaluate Large Language Models (LLMs) for classifying American Joint Committee on Cancer (AJCC) staging and American Thyroid Association (ATA) risk categories.
• To optimize the efficiency and accuracy of classifying thyroid cancer stage and risk.

Main Methods:

• Developed an NE framework including annotation guidelines, ground truth labeling, prompting strategies, and evaluation codes.
• Utilized four LLMs (Mistral-7B-Instruct, Llama-3.1-8B-Instruct, Gemma-2-9B-Instruct, Qwen2.5-7B-Instruct) for offline information extraction.
• Employed an ensemble-like majority-vote strategy for classification, validated on TCGA-THCA pathology notes and pseudo-clinical cases.

Main Results:

• The NE framework was developed and validated using 50 and 289 TCGA-THCA notes, respectively.
• An ensemble strategy achieved satisfactory performance in classifying AJCC staging and ATA risk categories.
• The developed framework and classifier demonstrated optimized efficiency and accuracy.

Conclusions:

• The proposed NE framework and LLM-based ensemble classifier effectively extract and classify critical clinical information for thyroid cancer.
• This approach enhances the accuracy and efficiency of determining AJCC staging and ATA risk categories.
• The study provides a valuable tool for analyzing large-scale thyroid cancer data.