WiNGPT-32B: An Open-Source, Locally Deployable LLM for RECIST Assessment via Chained Task Execution Using Radiology...

Area of Science:

• Oncology
• Radiology
• Artificial Intelligence

Background:

• Accurate tumor response assessment is crucial for cancer treatment efficacy.
• Longitudinal radiology reports contain valuable data for RECIST assessment.
• Existing LLMs may require specialized adaptation for medical tasks.

Purpose of the Study:

• To develop an open-source large language model (LLM) for Response Evaluation Criteria in Solid Tumors (RECIST) assessment.
• To utilize exclusively longitudinal radiology report text for training the LLM.
• To evaluate the performance of the developed LLM against GPT-4 and human radiologists.

Main Methods:

• Developed WiNGPT-32B, an open-source LLM, using knowledge distillation from GPT-4.
• Employed a Chained Task Execution (CTE) framework for modular RECIST assessment.
• Benchmarked WiNGPT-32B against GPT-4 and radiologists using a consensus standard.

Main Results:

• WiNGPT-32B achieved a 0.934 lesion extraction rate, surpassing GPT-4.
• For RECIST classification, WiNGPT-32B reached 0.805 accuracy, significantly outperforming GPT-4 (0.699).
• WiNGPT-32B demonstrated a superior F1 score (0.841) for progressive disease detection compared to GPT-4 (0.755).

Conclusions:

• WiNGPT-32B demonstrates the feasibility of text-only LLMs for longitudinal RECIST assessment.
• The CTE framework enables structured and modular RECIST evaluation.
• WiNGPT-32B shows promising performance, particularly in identifying disease progression.