Augmenting LLM with Prompt Engineering and Supervised Fine-Tuning in NSCLC TNM Staging: Framework Development and...

Area of Science:

• Artificial Intelligence in Oncology
• Medical Imaging Analysis
• Natural Language Processing for Clinical Data

Background:

• Accurate TNM staging is crucial for non-small cell lung cancer (NSCLC) treatment and prognosis.
• Current TNM staging faces challenges in standardization and interpretation due to reporting inconsistencies.
• Traditional NLP methods for staging are limited by manual rules and variability.

Purpose of the Study:

• To develop and validate an AI framework for robust and efficient TNM staging in NSCLC.
• To enhance the GLM-4-Air large language model using prompt engineering and supervised fine-tuning (SFT).
• To create an operationally efficient AI solution for clinical application in NSCLC staging.

Main Methods:

• Curated dataset of 492 real-world NSCLC imaging reports with physician-validated TNM annotations (AJCC 8th edition).
• Iterative prompt engineering with chain-of-thought and domain knowledge injection for GLM-4-Air.
• Parameter-efficient SFT using LoRA for T and N staging, creating a hybrid model.
• Evaluation on a held-out test set, benchmarking against GPT-4o, including clinical impact analysis.

Main Results:

• The hybrid GLM-4-Air model achieved superior staging accuracies: T 92%, N 86%, M 92%, overall 90%, outperforming GPT-4o (T 87%, N 70%, M 78%, overall 80%).
• Macro-average F1-scores for the model were T 0.914, N 0.815, M 0.831, exceeding GPT-4o's scores.
• Significantly reduced severe Category I staging errors, with zero errors in M staging and fewer in T/N staging.
• Demonstrated practical deployability with efficient inference on consumer-grade hardware.

Conclusions:

• The hybrid AI framework provides a highly accurate, clinically reliable, and cost-efficient automated NSCLC TNM staging solution.
• Domain-optimized smaller models, like the enhanced GLM-4-Air, show significant potential compared to generalist models.
• This approach holds promise for improving diagnostic standardization in resource-constrained healthcare settings.